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FOREWORD 


The aim of this book is to present in a widely appealing 
form the subject matter and methods of operations 
research (OR), a science acquiring in the latest years 
ever wider applicational arenas. It is of a relatively 
recent origin and so far its boundaries and contents 
can hardly be deemed rigorously defined. 

As a taught discipline, it is included in many curricula 
though what is covered is not always the same. In 
professional quarters, it lacks unanimity in much the 
same way. Some OR professionals treat it merely as 
the mathematical methods of optimization such as 
linear, nonlinear and dynamic programming. Others 
on the contrary, do not include these techniques in 
OR, approaching it generally from the viewpoint of 
game theory and statistical decision making. A third 
group tends to deny that OR exists as an independent 
discipline preferring to include it with cybernetics— 
a term inappropriately’defined and therefore various¬ 
ly understood. There are even those who tend to 
endow OR with an enormously wide sense claiming 
it to be a key science. 

Time, obviously, will tell which direction this rela¬ 
tively young discipline will take, which of the methods 
generally accepted as its constituents will hold, 
and which will branch off into self-contained dis- 
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ciplines. A particular question that remains concerns 
the future relation between OR and systems theory, 
a widely discussed topic of recent times. 

Beyond doubt, however, is the point that whatever 
the area of activity may be—production planning and 
inventory control, transportation, military operations 
and weapon system development, personnel manage¬ 
ment, social services, health services, communication 
systems, computer networks and information systems, 
to name but a few—the problems they pose with ever 
increasing rate are similarly formulated, can be iden¬ 
tified by several features they have in common and, 
last but not least, can be solved by similar methods. 
These problems are therefore conveniently grouped 
under the common heading of OR problems. 

A typical situation giving rise to an OR problem is 
as follows. An undertaking (a system of actions) having 
a clearly defined objective may be organized in several 
alternative ways by making a decision based upon 
choosing from a set of possible alternatives. Each choice 
offers its own advantages and disadvantages, so that in 
a complex situation the decision maker might not be 
able to make a preferable option at once and quickly 
decide why he should prefer one alternative and not 
another. To clarify the situation and compare the 
alternatives in several aspects, OR suggests a series 
of mathematical operations. Their aim is to analyze 
the situation critically and thus prepare a decision 
for those bearing the responsibility for a final choice. 

Without being confined to an exclusive area of prac¬ 
tice, the discussion in the book will be focused on the 
methodological aspects which are common to all OR 
problems wherever they might appear. Therefore the 
main emphasis will be placed on such methodological 
treatises as problem formulation, model development, 
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and assessment of computational results, rather than 
on mathematical rigor. The experience collected in 
the field suggests that it is these evaluation-and- 
analysis steps (not manipulatory) skill] in transfor¬ 
mations and computations) that might present major 
difficulties to an inexperienced practitioner when he 
will try to implement the mathematical methods of 
decision making to practical problems of value. 

In writing this text the author employed her many- 
year experience in applying OR to various practical 
arenas. The standpoint, into which the experience 
would have inevitably crystallized, was attempted 
to be delivered in the simplest possible form. 

The relevant mathematics is simple and requires 
some expertise in probability theory. Whenever the 
text goes beyond this simplicity frame, the needed 
explanations are immediately provided. As to the 
crest points of main ideas and methodology, they are 
treated with the highest rigor, calling for the attention 
and intense thinking of the reader. 

The author tended to avoid as possible an enter¬ 
taining presentation since an inavoidable curiosity 
dressing' would believingly distract the reader from, 
rather than help him to, the point. It does not imply, 
however, that the author would stick to stiffly formal, 
vapid fashion—a notorious attribute of mathematical 
texts. Rather, the adopted spirit is fairly jovial and 
even slipping occasionally to formulations which are 
not perfectly correct and, hence, open to criticism. 
The excuse is that the book is devoted primarily to 
the practitioners who are novices to the subject, rather 
than to experts. Ample reservations, should they be 
made to perfect the rigor, would only repell the reader, 
making it hard for him to grasp the essence of the 
matter. 




10 


Foreword 


The chapters of the book are not equal in difficulty 
of comprehension and the amount of mathematics 
involved. The reader who would like to get a general 
acquaintance with the subject, problems and possi¬ 
bilities of OR may confine himself to attentive reading 
through Chapters 1 and 2 and initial sections of the 
rest chapters. Beyond these introductory chapters, 
an intelligent reader will find what to calculate, thus 
having a closer look at the methods of OR and making 
himself familiar with the relevant mathematics to 
feel at ease if met with it in specialized issues. 

Chapter 6 dealing with queueing theory considers 
a series of methodologies almost entirely absent from 
nonspecial literature; that is the reason for its com¬ 
paratively large size. 

The aim of the book is to provide the reader with 
a foundation upon which to build a further study 
of a particular topic. References which cover extended 
applications and more formal mathematical treatment 
of the subject matter are provided at the end of the 
book. 


E.S. Wentzel 




Chapter 1 

THE NATURE AND USE OF OPERATIONS RESEARCH 


1.1 Operations Research—What It Is 
and What It Does 

Now that the revolutionary influence of science and 
technology is felt almost everywhere, ever more weight 
is put by science on planning, management and control. 
There are many reasons for this. The modern technology 
grows more sophisticated, scales of ventures and their 
aftermath effects spread dramatically, automatic con¬ 
trol being implemented in many fields, all this calls 
for the analysis of complicated, objective-bound pro¬ 
cesses to be performed from the viewpoint of their 
structure and organization. The science is required 
to deliver recommendations of how such processes 
should be controlled in optimal (judicious) fashion. 
Far behind are those times when a correct and efficient 
control had been found by trial-and-error approach. 
Now that the losses due to possible errors may cost 
too much to be afforded, the control must be developed 
scientifically. 

Requirements of practice called for special scientific 
methods which can conveniently be grouped under 
a common heading, operations research (OR). Referring 
to this term we shall imply the application of quanti¬ 
tative, mathematical methods to prepare decisions 
bound to be made in all the fields of objective-bound 
human activity. 
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To get a more closer look at what we mean under 
decision, consider a venture headed towards a certain 
target. The person (or a group of persons) who has 
initiated the venture always has a set of alternative 
courses of action, say, to pick up tools and build up 
a process in some way or another, to allocate resources 
according to one or another pattern, etc. The decision 
is the alternative that the manager chooses as his 
course of action. Decisions, consequently, may be 
bad or good, thought over or hasty, well-founded or 
arbitrary. 

The need for decision making is as old as the hills. 
Even at times of wilderness a leader of a tribe going to 
hunt a mammoth has to make a decision on where to 
place a trap and how to place the hunters and what 
must their weapon be, etc. We also would make deci¬ 
sions in our everyday life without even noticing it. 
Leaving, for example, for the office in the morning, 
we would make a series of decisions: what to wear? 
should an umbrella be carried? where to cross the 
street more conveniently? which urban transport to 
pick? and so on. A manager responsible for a process 
is to arrive at a decision every time when he has alter¬ 
natives, say, in placement of manpower, assignment of 
a work order, to name only a few. 

Does this decision making mean that we are involved 
in operations research? Not at all. Operations research 
begins whenever one or another mathematical tech¬ 
nique is applied to substantiate the decisions being 
taken. Obviously, in simple situations decisions are 
taken without any mathematics invoked, simply by 
the sound judgement and expertise. To decide what 
to wear going out or where to cross the street mathe¬ 
matics is hardly needed and undoubtedly will not be in 
the future. An optimum decision is arrived at as if by 
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oneself, guided i»y the practical experience. If at 
times a decision made is not the very best, then what? 
Mistakes are committed to improve on them. 

The decisions we will be concerned with more in 
this book, however, are those heavily loaded with re¬ 
sponsibility. Let, for example, the public transpor¬ 
tation network be planned in a new town having its 
own -layout of concentration points such as factories, 
apartment blocks, etc. Obviously, certain decisions are 
to be made concerning how to trace the routes and 
what vehicles should use them, where to place the 
stops, and what should be the interval in the ordinary 
and rush hours, etc. 

These decisions are much harder to make and, what 
is more important, many things depend on them. 
Erroneously chosen they spoil life and have an adverse 
impact on business activity of a whole town. 
Certainly, in this situation a decision may well 
be prepared intuitively from experience and common 
sense (as is not infrequently done). Yet the decisions 
are more judicious if backed up by mathematical 
reasoning. The preparative computations may help 
to avoid long and costly search of a decision by trial 
and error. 

To expand on this, let us take another example. 
Let us imagine that we are entrusted with a large scale 
venture, say, to divert waters from northwardbound 
rivers to water-hungry arid areas of the south. Should 
we take an arbitrary, willful decision that might bring 
about adverse effects on a global scale, or better to make 
preparative computations with mathematical models? 
The dilemma could hardly be viewed as ambiguous— 
the multilevel computations are mandatory. 

‘Score twice before you cut once’ a proverb says. 
Operations research is the very mathematical scoring 
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of future decisions, helping to spare time, efforts, and 
resources, and to avoid blunders, which are no longer 
such as to improve anything—the cost of respective 
correction will be too expensive. 

The more complex, expensive and large in scale the 
designed system is, the less allowable in it are willful 
decisions and the more gain in importance scientific 
methods which, when implemented, provide an esti¬ 
mate of each decision’s consequences, help discard the 
unallowable versions and recommend the most success¬ 
ful ones. They help in assessing whether the available 
information is adequate to prepare a correct decision 
and, if not, then indicate what data should be addi¬ 
tionally collected. It would be extremely risky to 
be guided solely by intuition, i.e. experience and 
common sense. Modern science and technology evolve 
so fast that the experience may simply not have 
been acquired. To say more, the undertakings initiated 
are often unique and original. Thus, the experience 
may be virtually absent, and common sense, if not 
proven by calculation, will be deceptive. The 
calculations that make the process of decision 
making easier are the subject matter of operations 
research. 

We have mentioned already that OR is a relatively 
young science (the notion of “youth”, though, is relative 
in scientific quarters: many scientific disciplines have 
been nipped in the bud shortly after their appearance, 
failing to find an application). The name operational 
research 1 was first used in the years just before the 
outbreak of World War II to describe an approach 


1 The name ‘operational research’ is still preferred by Brit¬ 
ish scientists to ‘operations research’ which was coined in the 
United States— Translator's note. 
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taken by interdisciplinary groups of British scientists 
summoned to solve strategic and tactical problems for 
military management. The decisions that were pre¬ 
pared by the groups were primarily concerned with 
the application of weapons and distribution of forces 
and facilities among various targets. Similar prob¬ 
lems, though with a differently named approach, had 
been taken elsewhere, in particular, in the Soviet 
Union. After the war, this approach spread into 
a variety of practical fields: industry, agriculture, con¬ 
struction, marketing, social services, transportation, 
communications, health services, pollution control, 
etc. There hardly exists an area of human enterprise 
where mathematical models and OR techniques have 
not been implemented in some form or the other. 

To get a better insight into the specifics of the sci¬ 
ence, consider several typical OR problems. Intention¬ 
ally taken to represent various fields, they, although 
simplified in the setting, bring forth fairly well the 
main idea of OR scope and objectives. 

1. Transportation problem. A network of factories 
consumes certain types of raw materials produced at 
several sources. The sources are linked with the facto¬ 
ries each in its own way, by railway, waterway, 
motor road, or air, having its own cost per unit of 
transported load. It is required to design a supply 
layout, specifying which source should supply what 
raw material and in what amount, and such that the 
demand is satisfied at minimum transportation 
cost. 

2. Construction project planning. A section of motor 
road is being built. The construction involves a certain 
amount of manpower, construction equipment, repair 
shops of a known capacity, trucks, and so on. It is 
required to plan the project, i.e. to schedule the order 
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of works, distribute the equipment and manpower 
along the construction site, and plan the maintenance 
and repair works, so that the project is terminated 
in the shortest term. 

3. Seasonal sales movements. The entrepreneur of 
a marketing firm plans to establish a network of branch 
off stores to market a certain stock of consumer goods. 
1 ie is to decide on the number of retailing stores, their 
location, distribution of inventories and personnel 
at each of the stores so that the business turns out maxi¬ 
mum economic efficiency. 

4. Snow fencing of roads. In the Nordic countries 
snow storms present serious interference to traffic. 
Any road blocked by snow means losses for the econ¬ 
omy. For road fencing, there exist a few alternatives— 
to construct the road with a suitable profile, to set 
up fencing facilities beside the road, etc.—each with 
its own cost for construction and maintenance. The 
decision maker has at hand the data on prevailing 
wind directions and frequency and intensity of snow¬ 
falls. He is to decide which means of snow control 
will be best in cost-efficiency terms, i.e. what means 
of fencing should be assigned to each of the roads 
subject to the losses from snow blizzards. 

5. Search for enemy submarines. An antisubmarine 
warfare officer receives a message that an enemy sub¬ 
marine has appeared in an area that Coastal Command 
surveillance. A group of aircraft takes off to find and 
bomb the submarine. The raid is to be set up in a most 
rational fashion, that is the routes are traced and 
altitudes and a pattern of attack are chosen such as 
to complete the task with the greatest certainty. 

6. Sample tests in quality control. To guarantee 
the quality of produced items at a specified level a 
factory establishes a sampling test system. A sound 
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system requires that the testing procedure, that is 
i•housing of sample size, sequence of tests, sorting 
rules, etc., should be devised so as toj provide for 
the quality at a minimum cost. 

7. Localizing an epidemic outbreak. Reportedly, 
a communicable desease broke out in an area. It is 
decided to conduct a medical examination of the area 
population so as to reveal those infected and infective. 
The search is to be performed by a certain medical 
staff equipped with proper facilities. It is required 
to schedule the search, that is to decide on the number 
of stations, their positioning, sequence of examination, 
types of analyses taken, etc., so as to maximize the 
revealed percentage of infective agents. 

8. Library shelf organization. A large public library 
has on its shelves the books in increased demand, in 
medium demand, and those retrieved very seldom. 
The books can be rearranged on the shelves and be¬ 
tween store rooms and some demands can be readdressed 
to other libraries. The problem is to work out a retrieval 
system such as to satisfy the demand in the best 
possible fashion. 

The examples could be piled up further on, yet even 
the above suffice to reveal the common features of OR 
problems. Although taken from various applications, 
the common traits can be readily identified: each 
lime we have spoken of an undertaking striving 
for a certain objective. Each time certain conditions 
have been specified to describe the environment, say, 
the facilities placed at the disposal. And each time 
within the framework of the posed limitations we are 
to make a decision such that to bring the undertaking 
to, in a sense, a most profitable condition. 

Accordingly, solution of such problems has been 
developed on a general basis of the techniques and 


2 — 0432 
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mathematical instrumentation which taken together 
make up the methodology and tooling of OR. 

The observant reader might have noticed that not 
all of the above examples would require mathematical 
treatment when practically handled. For some of 
them, a “good” decision taken on a common sense 
basis would suffice. The trend, however, is to increase 
the part of problems which require mathematical treat¬ 
ment for a decision to be made. An especially large 
momentum the mathematical techniques gain where 
automatic control systems are introduced. A control 
system, if employed to guide a process, rather than 
merely reduce data and retrieve information, will be 
unsound if the process under control has not been 
mathematically modeled and studied prior to system 
implementation. The mathematical techniques to pre¬ 
pare decisions come to carry even more weight as scales 
and complexity of enterprises increase. Air traffic 
of a small airport, for example, can be easily handled 
by one expertised traffic controller; a large airport 
requires an automatic traffic control system which 
operates according to a set in algorithm. Preparation 
of such an algorithm always involves some computation 
analysis performed beforehand, i.e. operations research. 

1.2 Main Concepts of and Approaches 
to Operations Research 

This section is devoted to the terminology, main 
concepts, and principles of operations research in its 
scientific background. 

The term ‘operation’ will imply any undertaking 
or system of actions which is conducted under a com¬ 
mon scheme and headed toward a certain objective. 
Thus all the undertakings presented in examples 1 
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through 8 of the previous section may be treated as 
operations. 

An operation is^always a,controlled action, which is 
meant to say that we are to decide in what way to 
select the parameters setting up its structural organi¬ 
zation. The organization is to be understood here in 
a wide sense and includes the technical means involved 
in the operation. 

Any option, that is alternative set, of parameters 
which in their selection depend upon the will of the 
manager is referred to as a decision. Decisions can be 
.successful and unsuccessful, meaningful and unsound. 
The optimal decisions are those which are preferred 
to the others for one reason or another. The aim 
of operations research is to quantitatively prepare 
optimal decisions by finding optimal solutions for OR 
problems. 

At times, though it is a relatively seldom occasion, 
the study yields a sole, strictly optimal solution; still 
more often, it bounds an area of optimal solutions virtu¬ 
ally equal in value, within which a final option is 
to be made. 

Note that the decision making proper does not fall 
within the operations research domain. It is a manager 
or, what is more often, a group of persons who is 
entrusted with the final decision making and who 
boars the responsibility for the choice. In arriving 
at a choice, the decision makers may take into account 
other reasons (quantitative or qualitative in nature) 
not considered in preparing mathematical recommen¬ 
dations. 

Even with totally automatic systems of control 
which seem to make decisions with no human inter¬ 
ference the judgement of a human is present in the 
form of the algorithm employed by the system. It 
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should be remembed that the development of the algo¬ 
rithm or selection of one of its plausible versions is a 
decision making process too, indeed laden with re¬ 
sponsibility. With the progress in automatic controllers, 
the functions of the human are not taken up by a ma¬ 
chine, rather, they shift from a basic level to a more 
intelligent one. To add more weight to the argument, 
some automatic control systems are developed so 
that the human may actively interact to aid the pro¬ 
cess of control. 

The parameters whose collection forms the set of 
a decision are referred to as controllable or decision 
variables. These variables may materialize as various 
numbers, vectors, functions, physical characteristics, 
etc. To illustrate, if a transportation schedule is 
worked out for similar commodities from origins 
A u A 2 , . . ., A m to destinations B lt B it . . ., B n , 
then the decision variables are the numbers Xi jt indi¬ 
cating the amount of commodities from the ith origin, 
A i , to the ;'th destination, Bj. A set of numbers x n , 
X 12 , • . ., Xx n , • • •, X m l) %m 2 > • • •> ^mn forms a de¬ 
cision. 

Simple OR problems operate with a comparatively 
short list of decision variables. In a majority of prob¬ 
lems of practical value, though, the number of deci¬ 
sion variables may be very large, the fact in which the 
reader is invited to convince himself by trying to 
evaluate and label the decision variables in examples 1 
through 8 of Section 1.1. To make the discussion more 
simple we shall denote the set of decision variables 
by a single letter x and spell it ‘decision x'. 

In addition to decision variables which we may con¬ 
trol within certain limits, any OR problem contains 
specified, constraining conditions which must not be 
violated. Examples of such constraints may be capacity 
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of a truck, planned production target, weighting char¬ 
acteristics of equipment, etc. They in particular 
include the means (monetary costs, labour, equipment) 
that are put under the managerial disposal, or any 
other conditions limiting the decision. Taken as 
a whole, they establish what we would call a set of 
possible decisions. 

We denote this set again by a single letter X, and 
write down the fact that x belongs to the set symboli¬ 
cally as x £ X, read as element x belongs to set X. 

Restating our goal in mathematical terms, we are 
to evaluate on a set of possible decisions X those of 
them x (a single one, or more frequently a region of 
decisions) which are in one or several aspects more effi¬ 
cient, i.e. more useful or preferable to, than the others. 
To judge the merits of particular decisions and com¬ 
pare them, a quantitative criterion is introduced, 
called the measure of effectiveness and in practical 
problems can also be referred to as the objective 
function. This performance index should be chosen 
such that it describes the objective of the operation 
under concern. A decision is deemed best when it 
perforins upper mostly in arriving at the objective. 

In deriving a measure of effectiveness, W, we first 
of all ought to ask ourselves what we are striving at, 
embarking on the enterprise. Choosing a decision, we 
naturally would prefer one that maximizes the effec¬ 
tiveness function, or keeps it at a minimum. Obviously, 
the profit of a venture should be made a maximum; 
when the measure of effectiveness is built around costs, 
it should be minimized. Mathematically we shall 
describe it as W => max if the function is to be maxi¬ 
mized, and as W =>- min if it is to be minimized. 

In those situations where the outcome of an operation 
comes under the influence of random factors—these 
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might be weather, variations in demand and supply, 
equipment failures, and so forth—the measure of 
effectiveness is derived as the mean value (mathemati¬ 
cal expectation) of the function which is going to be 
minimized (maximized). (In more detail such situa¬ 
tions shall be discussed in Section 2.2.) 

When random factors influence the operations whose 
objective, A , can he either made or not made at all 
and none of the outcomes in between is of interest, 
the measure of effectiveness is chosen in terms of the 
probability to make this goal, P (A). Shooting a tar¬ 
get, for instance, on an indispensable condition to 
demolish it, the measure of effectiveness will he the 
probability of demolishing. 

Setting the index of performance in a wrong way can 
entail adverse consequences. Operations conducted 
with an improperly chosen measure in view might 
incur unjustified costs and losses. 

To illustrate how the measure of effectiveness 
should be derived we come back again to examples 1 
through 8 of Section 1.1. Choosing a natural measure, 
we indicate whether it should he minimized or maxi¬ 
mized. Tt should he kept in mind, however, that 
not in all of the examples a measure of effectiveness 
is uniquely brought to the surface by the wording 
of the problem. Consequently, the reader may deviate 
in his judging the situations from the viewpoint given 
below. 

1. Transportation problem. The problem is to supply 
raw materials at minimum transportation costs. The 
measure of effectiveness, /?, is equal to the total trans¬ 
portation costs per unit time, say, per month; R =>- 
=»- min. 

2. Construction project planning. The construction 
is to be planned so that it is completed in a shortest 
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possible period. It would be natural to choose the 
period of construction as a measure of effectiveness 
if it were not for random impacts (equipment failures, 
delays in completion of certain works, and so forth). 
The measure, therefore, may be the mean expected 
time, T, of completion of the project; T =>■ min. 

3. Seasonal sales movements. For a measure of 
effectiveness, one may take the mean expected profit 
G, gained from the sales over the season; G=>-max 9 . 

4. Snow fencing of roads. The problem is to devise 
the most economical snow fencing project. Hence, 
the measure of effectiveness may be chosen as an 
average for a period (say, for year) of combined costs, 
R, for maintenance and repair of the roads, including 
in that index the basic costs for erection of fences, 
snow removal costs, and costs due to delayed traffic. 
Obviously, R =£- min. 

5. Search for enemy submarines. The objective 
of the raid is perfectly certain—to sink the submarine; 
therefore, a measure of effectiveness should be the 
probability that the submarine will be eliminated. 

6. Sample tests in quality control. A natural mea¬ 
sure of effectiveness brought to the surface by the 
problem formulation would be the mean testing costs, 
R, per unit time, provided that the system of tests 
guarantees a specified quality level, say, that the 
mean rejection percentage is not above a specified 
target value; R =$» min. 

7. Localizing an epidemic outbreak. An adopted 
measure of effectiveness may be the average percentage 
(fraction) Q of the infected and infective people that 
have been evaluated; Q =*- max. 

2 Henceforth we shall denote a mean value hy an overhead 
bar. 
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8. Library shelf organization. An intently made 
slip in the formulating of the problem brings some 
spirit of alternative in finding the appropriate measure 
of effectiveness. The invitation to satisfy the demand 
in the best possible fashion does not lead to a straight¬ 
forward measure. Judging the quality of the service 
by the time spent in waiting for a book, the measure 
may be the mean time T of waiting, that is, T =>- min. 
Taking another tack, one may choose as a measure 
of effectiveness the mean number of books, M, deliver¬ 
ed per unit time; M=*-max. 

The considered examples were purposefully selected 
to be simple enough to make the choice of a measure 
of effectiveness comparatively straightforward and 
following obviously from the problem formulation 
because of its, almost in all cases, unique objective 
orientation. In practice, however, it is not always the 
case. The reader would readily find this out by trying, 
for instance, to choose a measure of effectiveness for 
an urban transportation system. What should it be: 
an average speed of moving passengers through the 
town? or an average number of handled passengers? 
or perhaps an average mileage a person has to walk 
because the transport fails to bring him to a wanted 
place? The problem suggests a lot to think over. 

Unfortunately, the choice of a measure for the 
majority of applicational problems is not an easy 
task and no ready-made approach can be recommended. 
It is typical for a problem of the sort that offers some 
difficulties that a sole number cannot serve as an all- 
inclusive measure, so other numbers have to be sought 
to properly describe the effectiveness. We shall look 
more closely at such multiple criterion problems 
in Section 2.3. 
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1.3 Model Development 

Preparing to accommodate quantitative methods of 
study, any field requires a mathematical model. To 
develop a model, a real phenomenon (an operation in 
our case) is inevitably simplified into a scheme which 
then is described invoking a suitable mathematical 
technique. The simple yet still sufficient is the 
developed model, i.e. the better it describes the 
phenomenon under study, the more successful will 
be the study and the more useful the resulted recom¬ 
mendations. 

No general approach exists to formulating mathe¬ 
matical models. For any particular situation, a model 
is developed according to the type of the operation 
under concern, its objective, and the task of the study 
(i.e. reflecting the parameters which are to be defined 
and the impacts to be included). The accuracy and 
elaboration of a model should be validated against 
(i) the accuracy required from the solution, and (ii) the 
data that is at hand or can be additionally collected. 
If the input data of the computation does not possess 
a required accuracy, then obviously it would be sense¬ 
less to go into details, construct an elaborate model 
and waste time (of both the analyst and a computer) 
to achieve a precisely optimized solution. Unfortunate¬ 
ly, this principle is often overlooked and the models 
employed are superfluously elaborated. 

The mathematical model should involve all the 
important facets of the phenomenon being studied 
and all the critical factors on which mainly hinges 
the success of the operation. On the other hand, a model 
should be possibly simple and not stuffed with a lot 
of minor factors whose account complicates the analysis 
and makes the results hard to grasp. Two traps always 
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await a mathematician on his way to a model; first, 
he might flounder in details, or, second, he might 
adopt too coarse an approximation leaving the essence 
of the action beyond the model. The trade of model 
development is an art indeed with the expertise being 
acquired only gradually as it would in arts. 

Since a mathematical model often does not .immedi¬ 
ately follow from the problem statement, it would be 
a good practice not to blindly believe in the infalli¬ 
bility of a single model, rather, it would be wise to 
compare the results yielded by a variety of models so 
as they may compete. To be sure in a solution, a prob¬ 
lem should be attacked several times, from various 
angles, i.e. with various allowances, technology, and 
models. If the analytical inference changes from one 
model to the next only slightly, it is a strong argu¬ 
ment in favor of the study having been conducted 
objectively. On the contrary, if the inferences of 
different models deviate drastically, the conceptual 
framefork of the models should be revised, looking 
upon which of the models describes the action more 
adequately, and then, if needed, run a test experiment. 
Typical for an OR analysis is also that a recursive 
tack is employed to correct the model after a compu¬ 
tational run before the next. 

Model development is the most important and crit¬ 
ical stage of any study; it requires deep insight and 
understanding into the phenomena being modeled 
rather than maturity in relevant mathematics. As 
a rule, “pure” mathematicians who have not managed 
to consult experts in modeled system trade are able 
to handle model development only poorly. They would 
place the main emphasis on the relevant mathematical 
tooling rather than on the practical side of the prob¬ 
lem. 
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The collected experience has made evident that the 
most successful models have been developed by either 
the specialists in the field who have been at ease with the 
appropriate mathematical techniques or the teams 
recruited of practitioners in the field and mathemati¬ 
cians. Practitioners in many fields, engineers, biolo¬ 
gists, medical experts to name only a few, find it 
very useful to consult a mathematical expert in OR 
when they meet with a necessity to make decisions 
on a scientific basis. These consultations are advanta¬ 
geous to both sides as the mathematicians get acquainted 
with practical problems from diverse trades. To tackle 
them mathematicians often have to boost their exper¬ 
tise and extend, generalize and modify the known 
techniques. 

The mathematical education of a practitioner who 
would like to perform an independent OR study in its 
own field should involve a variety of mathematical 
disciplines. In addition to a classical college course 
of mathematical analysis, OR often employs modern 
and relatively recently evolved techniques such as 
linear, nonlinear, and dynamic programming, game 
theory, decision making, queueing theory, and some 
others. This book is supposed to supply the reader 
with a certain grasp in these techniques. 

An especial attention among the educational disci¬ 
plines should he devoted 1o probability theory: easy 
operating with statistics and probability patterns 
is to be preferred to an academic comprehension. The 
stress on this discipline can be explained by the fact 
that, the majority of operations has to be conducted 
under the conditions of incomplete certainty, when 
the performance and outcome depend upon random 
factors. Unfortunately, it is a rare occasion in engineer¬ 
ing, biological, or medical quarters that a specialist 
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has a firm and easy comprehension of probability. The 
concepts and rules of the theory find only formal appli¬ 
cation with no due perception of their sense and spirit. 
It is infrequent that the theory is viewed to be a sort 
of magic wand yielding information from nothing, i.e. 
from total ignorance. Those who think so are under 
a misapprehension since probability theory is used 
but to transform data on observed phenomena to infer 
the behaviour of those which cannot be observed. The 
reader is supposed to possess the basic knowledge in 
the theory. 

With the above list of mathematical disciplines 
involved we have not the smallest intention to scare 
the reader off. First, all things are difficult before 
they are easy, and any instrument can be mastered 
if the motivation is strong enough. Second, not all 
the techniques need be applied to a problem at once; 
they can be learned one by one starting with that 
which is needed most. An answer to what problems 
call for what knowledge can be again obtained from 
this book. 

In developing a model, the complexity of mathemat¬ 
ical tools may be different depending on a type of 
operation, objectives of the study and accuracy of 
input data. In the simplest situations algebraic equa¬ 
tions can suffice. If they are more involved, say, calling 
for a phenomenon to be considered in its dynamics, 
then differential equation (ordinary or partial) tech¬ 
nique is implemented. In most intricate cases, when 
both running an operation and its outcome depend 
on a large number of intimately interrelated random 
factors, the analytical techniques fail altogether and 
the analyst has to employ Monte Carlo methods of 
statistical modeling (discussed in Chapter 7). Loosely 
speaking, the idea of this approach may be described 
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as follows. A computer simulates the process of an 
operation development with all the random variables 
involved. This manipulation of the process yields 
an observation (realization) of one random operation 
run. One such realization gives no grounds for decision 
making, but, once a manifold of them is collected after 
several runs, it may be handled statistically (whence 
the term statistical simulation) to find the process 
means and make inferences about the real system and 
how, in the mean, it is influenced by initial conditions 
and controllable variables. 

Both analytical and statistical models are widely 
implemented in operations research. Each of the models 
possesses its own advantages and disadvantages. The 
analytical models are more rough, account for lesser 
number of variables and are always in need for some 
allowances and simplifications. On the advantage 
side, though, they yield more meaningful results 
which better reflect the relevant regularities. Above 
all, however, they are better amenable to search of 
optimal solutions. 

The statistical models are more accurate as compared 
with the analytical, they are also more elaborated, 
do not require those robbing assumptions, and can 
account for a large (theoretically unlimited) number 
of factors. Yet, these methods possess their own disad¬ 
vantages, they are bulky, poorly analyzable, need 
a lot of computational time, and, above all, are very 
difficult at optimal solution search which is performed 
by trial and error. 

The novices in the field whose expertise in OR studies 
is comparatively small often start a study with, not 
indispensable, constructing a statistical model, making 
effort to incorporate as many factors as available. They 
forget a simple thing—getting a model and running 
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the computations is but a half way to a useful result; 
more important is to succeed in analyzing the com¬ 
putational printouts so that they might be translated 
to the rank of recommendations. 

Best works in the OR field were based on the joint 
application of analytical and statistical models. The 
analytical model helps to generally perceive a phe¬ 
nomenon, and outlay, as it were, a contour of major 
regularities. Any further improvements can be achieved 
with statistical models (in more detail, see the topic 
in Chapter 7). 

To conclude, we are bound to say a few words about 
the so-called simulation theory. It is applied to the 
processes whose performance is to he directed from 
time to time by intervening human will. The manager, 
or management, in charge of an operation is to make 
decisions depending on the situation being reported 
similar to that how a chess player analyzing the game 
chooses a move. The decision is then computed over 
by the mathematical model to evaluate the system’s 
response to this decision in a bit of time. The next 
decision will be made already with a due account of 
a new real state of the system, and so on. As a result 
of the manifold repetition of the procedure the manager 
as if gains the expertise, and by improving on his 
own and others’ mistakes comes gradually to make 
correct decisions, even if not exactly optimal, then 
somewhere close to that. These procedures having 
come to be known as operational gaming became recently 
very popular and widely recognized as a valuable tool 
in operational training of managerial staff. 




Chapter 2 
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APPROACHING OPERATIONS RESEARCH PROBLEMS 


2.1 Two Ways of Posing the Problem. 

Deterministic Models 

An operations research problem can be posed in either 
of two ways. We shall call them, accordingly, primal 
and inverse. A primal problem answers the question: 
what will be if under specified conditions we make 
a decision x £ X? In particular, what will amount 
with this x the value of a chosen measure of effec¬ 
tiveness, W, or a set of such measures? 

To solve such a problem the researcher develops 
a mathematical model which represents one or several 
measures of effectiveness as functions of specified 
conditions and controllable variables. 

An inverse problem answers another question: how 
to find such a solution x that the measure of effective¬ 
ness, W, be a maximum? 1 

Basically, primal problems are simpler to solve than 
inverse. The latter obviously cannot be solved, without 
first solving the primal. Some type of operations evolve 
an easily solved primal problem so that it requires 
no special attention. In other operations, the model 
development and measure of effectiveness com¬ 
putation are fairly nontrivial, as, for example, is 


1 In the following we shall always discuss the maximum 
search situation whatever it may be. Obviously, the minimum 
case becomes that of maximum by simply changing the 
sign of W, i.e., min W — max (— W). 
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the case with queueing theory we shall discuss in 
Chapter 6. 

A few more details on the inverse problem are in 
order. If the number of feasible solutions constituting 
the set X is small, then we may simply evaluate the 
value of W for each of them to compare and pick up 
that one (or those) which maximize W. Hence, an 
optimal solution is found by exhaustive search. When 
the number of feasible solutions constituting the set 
of possible decisions X is large, however, a blind search 
by exhaustion is cumbersome, and often virtually 
impossible. To deal with such situations, the researcher 
has to invoke direct search methods. These find an 
optimal solution by successful trial approximations 
with each next outcome appearing closer to the sought 
optimum than the previous. We shall discuss some of 
these techniques in Chapters 3 and 4. 

At the moment we confine ourselves to formulat¬ 
ing the problem of finding the optimum solution in 
its most general form. 

Assume that we are conducting an operation, whose 
success we can influence by making a decision x in 
one way or another (as will be recalled, x is a group 
of parameters rather than a number). Let the effective¬ 
ness of the operation be defined by a single index, 
W =4- max. 

Assume further that we deal with the simplest, so- 
called deterministic, situation when all the conditions 
of the operation are known beforehand, i.e. no uncer¬ 
tainty is involved. Then all the variables on which 
the performance of the operation depends may be 
broken down into two groups: 

(i) those specified beforehand and thus known 
(conditions on operating the action); we shall 
shortly denote them by the Greek letter a; 
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(ii) those under our control, constituting collectively 
a solution x. 

Note that the first group of variables contains, among 
others, the constraints (occasionally referred to as 
restrictions) imposed on a solution, i.e. defines a region 
of feasible solutions X. 

The measure of effectiveness W depends on variables 
of both groups, or putting it as a formula 

W = W (a, x) (2.1-1) 

Considering Eq. (2.1-1) it should not go unnoticed 
that both x and, generally, a, are collections of values 
(i.e. they are vectors), functions or similar, rather 
than numbers. Commonly present among the specified 
conditions a, the constraints imposed on controllable 
variables assume the form of equations or inequalities. 

Assuming that the form of Eq. (2.1-1) is known, that 
is the primal problem has been solved, we may for¬ 
mulate the inverse problem as follows. 

Given a set of conditions a, find a solution x — x* 
such that maximizes the measure of effectiveness W. 

We denote this maximum by 

W* = max {W (a, z)} (2.1-2) 

5C£X' 

which is read as: W* is the maximum value of W (a, x) 
taken over all solutions belonging to the set of all 
possible solutions, X. 

We should accustom ourselves to the equations of the 
type. They will be of help in the following (in Chapt¬ 
er 4, for instance). 

Now, what we have got is a typical problem on 
searching a maximum of a function or a functional. 2 

2 It should be recalled that the functional is a mathematical 
quantity which depends upon a function. When a solution x in¬ 
cludes functions as well as numbers, then W (a, x) is a functional. 


.' 1-0432 
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It belongs to the well developed class of variational 
problems of mathematics. The easiest of them (maxi¬ 
mum and minimum problems) are familiar to any 
graduated engineer. Seeking a maximum, or minimum 
(an extremum, to be concise), of a function of many 
variables we commonly differentiate it with respect 
to all the variables (controllable variables in the dis¬ 
cussed circumstances), set the derivatives to zero, and 
solve the resulted set of equations. Seemingly, nothing 
could be more simple. Yet, in operations research this 
classical approach works poorly for several reasons. 
First, when the variables are many, solving the relevant 
set of equations appears often a more complex task 
than the direct search for an extremum. Second, with 
some constraints imposed on controllable variables, 
an extremum more often than not lies at the boundary 
of the region X rather than at the point of zero deriva¬ 
tives. This brings about specific difficulties of the multi¬ 
dimensional variational problem under constraints, 
whose complexity might occasionally render it not 
amenable to a modern computer. Besides, in some 
problems the function W has no derivatives at all—it 
is specified, say, only for integer-valued arguments. 
All the mentioned circumstances make the search of 
extremum not so simple a procedure as it might seem 
at first glance. 

The strategy of the search for extremum and the 
related optimal solution x * should always be devised 
paying due regard to the form of both the function W 
and imposed constraints. To exemplify, if IT is a linear 
function of controllable variables, x x , x a , . . ., x n , 
and the constraints imposed on the variables have the 
form of linear equations or inequalities, then we arrive 
at a linear programming ( LP ) problem which is solved 
by comparatively simple and, what is more important, 
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standard procedures (see Chapter 3). If W is convex, 
special techniques of convex programming are employed, 
in problems where the only nonlinearities are qua¬ 
dratic, quadratic programming is invoked (see 18]), 
To optimize the control of multistage activities, 
dynamic programming techniques are implemented (see 
Chapter 4). Finally, there exists a family of numerical 
methods devoted to the search of extrema; they typi¬ 
cally use algorithms and are therefore well suited for 
implementation on a digital computer. Some of them 
involve a random search procedure which not infre¬ 
quently appears more efficient than that of ordered 
exhaustion for the case of multidimensional problems. 

Summarizing, we can say that the search for optimal 
solution in the simplest, deterministic environment 
is a purely mathematical problem of the calculus of 
variations variety (both constrained and unconstrain¬ 
ed), which may offer computational rather than prin¬ 
cipal difficulties. The procedure is not that straight¬ 
forward, however, when the problem involves un¬ 
certainty. 


2.2 Decision Making under Uncertainty 

In the previous section we considered the optimization 
problem of OR for the deterministic case, i.e. for the 
measure of effectiveness depending on two groups of 
factors, those specified and known beforehand a, 
and controllable variables, x. More often than not 
real problems involve another group, that of unknown 
factors which in their generality we will denote by 
a single symbol |. Now, the measure of effectiveness W 
depends on all the three groups, or symbolically 

W = W (a, z, £) (2.2-1) 


3 * 
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Since W depends on unknown factors |, it cannot 
be computed explicitly even though a and x arc given. 
Hence, the search for optimal solution is no longer 
certain. We cannot after all maximize an unknown 
quantity. Still, we cannot help making this quantity 
possibly great. Some people do succeed in the circum¬ 
stances when the entire situation is not obvious, don’t 
they? Occasionally they do indeed. Converting this 
into mathematical language, we pose the following 
problem. 

Given the conditions which must be satisfied, a, 
and allowing for the unknown factors |, find 
a solution x £ X such that the measure of effective¬ 
ness, W, be possibly large. 

This formulation can no longer be termed rigorous, 
indeed it contains a proviso “possibly”. The presence 
of uncertain factors translates the problem onto a new 
level where decisions are to be made under uncertainty. 

Let us have a closer look at the new problem. First 
of all, we ought to confess that uncertainty carries with 
it no advantage. If the conditions of operations are 
not known we can no longer optimize the decision 
with that degree of success we could if adequate 
information were at hand. Therefore any decision made 
under uncertainty is worse than that made in the con¬ 
ditions known beforehand. Anyway, we are bound to 
make a decision, whether it be good or bad. Our task 
then is to supply the decision with as much judgement 
as possible. Thomas L. Saaty, an outstanding expert 
in OR, has his reason putting a bit of irony in his 
definition of the trade as the art of giving poor answers 
to practical questions which other methods answer 
still worse [1]. 

We are confronted with the necessity of making 
decisions under uncertainty at every step. For example, 
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bound for vacation trip we are packing a suitcase. 
Its dimensions along with the weight of the packing 
and the set from which we are to choose the things 
are given (conditions a), and the weather at the desti¬ 
nation place is not known beforehand (conditions |). 
Which clothes ( x ) should we take along? This problem, 
although similar outwardly to those of OR, will be 
handled, certainly, with no recourse to mathematics, 
though we shall invoke some statistical data, say, 
on the most probable weather in the area and our own 
tendency to, e.g. catching cold. Whether consciously 
or subconsciously, we somehow optimize the decision. 
Interestingly, people seem to employ for that different 
measures of effectiveness. A young person would 
rather tend to maximize the sum of expected pleasures 
(let us put aside how this can be estimated in a quan¬ 
titative fashion), whereas an elderly traveller would 
more likely minimize the probability of catching 
cold. 

Consider now a more serious problem. A selection of 
goods is planned for sales. The salesman would like to 
maximize the profit. However, he cannot tell in 
advance how many customers will be shopping and 
what each of them will need. What will he do? There 
is no escaping uncertainty, and yet a decision has to 
be made! 

Another example. A dam is planned to be built to 
protect a region being flooded. Neither moments of 
floods, nor their water levels are known beforehand. 
Yet the project must be planned and no uncertainty 
can relieve us of the necessity to do it. 

Finally, an even more complex problem. A weaponry 
development is planned over a several years horizon. 
Neither a particular opponent, nor his weapon at the 
time is known. A decision, however, must be made. 
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So that to take such decisions judiciously rather 
than at random, by rush of inspiration, modern 
science suggests several methods. Which of them to 
adopt depends on what is the nature of unknown fac¬ 
tors what is their origin, and who controls them. 
Putting it another way, with what type of uncertainty 
we meet in a particular study? 

The reader can reasonably ask if it is really possible 
to impose type qualification on uncertainties. As turns 
out, it may be done. 

First of all consider the uncertainty of the, so to say, 
most favorable type for the study. This turns up when 
the unknown factors belong to the domain of proba¬ 
bility theory, that is, are random variables, or func¬ 
tions, whose statistical characteristics are known or can 
in principle be obtained by an appointed time. We 
shall refer to these OR problems and relevant uncertain¬ 
ty as stochastic. 

We should better demonstrate this type of problems 
by means of an example. Let us imagine that we re¬ 
organize the work of a canteen with the intention to 
enhance its throughput. No exact data are given about 
how many customers will visit it over working hours, 
when they will appear, what meals they will order, 
and how long each of them will be served. However, 
the required characteristics, if not available so far, 
can he obtained by statistical inference. 

Another illustration. It deals with the design and 
operation of a vehicle repair and overhaul facility. 
The objective is to cut down idle time due to break 
down in service and repair. The'facility is to handle 
the unanticipated repair, hence, the failures, time 
in repair or overhaul are all factors of random nature. 
However, the characteristics of all random variables in¬ 
volved can be gained from the respective statistical data. 
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We get a better sense of this “favorable” uncertainty 
if we consider it in still more detail. Let the unknown 
factors be random variables whose distribution den¬ 
sities, mathematical expectations, variances, or other 
statistical characteristics are principally known. 3 Then 
the measure of effectiveness, a function of these char¬ 
acteristics, will itself be random. A random variable, 
however, cannot be maximized since it remains random 
at any option x, that is it cannot be put under control. 
What course will we choose? 

A surface idea that comes to mind first is to sub¬ 
stitute mean values (expectations) for the respective 
random factors £. The problem then becomes deter¬ 
ministic and can be solved by ordinary techniques. 

To say least, the approach is fairly tempting and 
in some situations even justified. Indeed, the majority 
of applicational problems in physics, mechanics, and 
engineering is at every step tackled along these lines, 
neglecting the random nature of some parameters 
involved (heat capacity, inductance, friction coeffi¬ 
cient, etc.) in that their mean values are employed. The 
point is essentially how much random these param¬ 
eters are; if they deviate of their means only slightly, 
we may use the means and be justified to do so. It is 
like this in operations research: we may neglect ran¬ 
domness in some problems. For instance, if we are 
engaged in planning raw material supplies for several 
production facilities (see Example 1 in Section 1.1), 
we may, to a first approximation, neglect, say, that 
real output of supply sources may randomly fluctuate 
(provided that the bulk supply is well organized). 
The same obviation of random variables by their 


3 If among the variables of | are stochastic functions, they 
can be converted to series of random variables. 
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expectations will be inappropriate, however, if the 
randomness can substantially impact the outcome of 
operation. To illustrate, we take a most primitive 
example of shooting a target. If after making several 
shots resulted in randomly placed hits we took instead 
their mathematical expectation, i.e. the target itself, 
we would be guaranteed to hit the target with each 
shot which is obviously wrong. 

Another example with a lesser evidence. It deals 
with the design and operation of a vehicle repair 
facility that is to service, e.g., a government motor 
pool or a taxi cab company. Should we not plan the 
randomness of breakdowns (i.e. change the random 
time of failure-free operation for its mathematical 
expectation) and the randomness of repair time, then 
the capacity of the shop would simply not he up to the 
demand (the full evidence of that will be presented in 
Chapter 6). Recurrent, though, are operations where 
randomness is basically involved so that there is no 
driving them to a deterministic end. 

Consider now a situation in which the factors | are 
“substantially” random and therefore markedly in¬ 
fluence the measure of effectiveness which, hence, is 
substantially random, too. 

An intuitively sound approach is to operate with 
the mean of the random measure, i.e. take W = 
E [W] and choose a decision x such that the averaged 
measure be a maximum, viz., 

W = E[W (a, x, |)1 => max (2.2-2) 

Note that this is the very approach we have adopted 
in Section 1.2 when we took mean profit and mean 
time rather than simply profit and time as a measure 
of effectiveness for problems with uncertainty. For 
the majority of situations, this approach (we should 
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call it ‘optimization in the mean’) is perfectly justi¬ 
fied. Really, there is more judgement in a solution 
chosen so as to maximize the measure of effectiveness 
in the mean than in one taken at random. 

We may ask after all how far have we succeeded in 
obviating the uncertainty? Certainly, to some extent 
it is still present. The effectiveness of each operation 
performed with particular values of random factors £ 
can markedly deviate from that expected, assuming 
both larger and, unfortunately, lesser values. We 
may only content with the thought that the payoff 
after multiple repetitions and the optimization in the 
mean will be larger than that yielded with no compu¬ 
tations at all. 

This optimization in the mean is often used in 
practical stochastic OR studies with no investigation 
into its validity. To be valid, however, the operation 
to which it is applied must be of a recurrent variety so 
that a loss in the measure of effectiveness incurred in 
one situation might be compensated by its surplus 
in another. To illustrate, if we perform a long series 
of similar operations with the objective to maximize 
the profit, then profits of the operations are to sum 
up so that a minus in one operation can be met by 
a plus from another with the positive net result. 

Yet those who think that this condition will always 
guarantee the safe side are far from being true. To 
prove, consider an example of automatic control 
system planned to manipulate the ambulance car 
fleet in a big city. The calls from various districts 
arrive at random at the central control board to be 
retransmitted to particular, local, ambulance stations 
from where cars will he sent. Tt is required to develop 
a dispatching algorithm such that will enable the 
ambulance services to operate with the best efficiency. 
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To this end we need first of all to devise a measure of 
effectiveness, W. 

Obviously, the time a patient waits for the doctor is 
desired to be possibly short. However, this time is 
a random variable. Should we apply the optimization 
in the mean, then we need to choose the algorithm 
that minimizes the mean expectation time. If we 
adopt this approach, some of our patients would run 
into real trouble, since the expectation times cannot 
be added—too long waiting of one patient cannot be 
compensated for by a quick serving of another. Hence, 
if we adopted mean waiting time T as a measure of 
effectiveness, we would tend to prefer the algorithm 
which, although watching for short mean waiting 
time, might agree with very long waiting times for 
some of the callers. To avoid these unagreeable cir¬ 
cumstances we can augment the measure of effective¬ 
ness by a requirement calling for the real expectation 
time T be not above some largest value t 0 . Since T 
varies at random we may not boldly require that the 
condition T^.t 0 be satisfied; we may only require 
that it be satisfied with such a high probability as to 
make the event T ^ t„ virtually true. Proceeding in 
this way, we can adopt the value for the probability, 
P, which is close to unity, say 0.99 or 0.995, rendering 
the event practically true. Now we can require that 
the condition T ^ t 0 be met with a probability not 
lesser than p, or 

P (T < f 0 ) > p (2.2-3) 

This constraint implies that the region of feasible 
solutions X excludes those of them which fail to satisfy 
the condition. The constraints of the type (2.2-3) will 
be referred to as stochastic. If present, they substan¬ 
tially complicate the optimization. 
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The researcher should be particularly cautioned in 
applying the optimization in the mean to a solitary, or 
even unique, operation rather than recurrent. The 
applicability in such situations depends on what re¬ 
sults from the failure of the operation, i.e. a low value 
of effectiveness index brought about; occasionally 
it may imply a catastrophe. What reason to hope for 
large payoff in the mean if this particular action might 
turn out bankruptcy. Again, these catastrophic results 
may be avoided by introducing stochastic constraints. 
At a substantially large confidence coefficient p we 
may be practically guaranteed of not being ruined. 4 

Now that we have shortly considered the case of 
“favourable” (stochastic) uncertainty and .generally 
elucidated the relevant optimization we are still 
halfway to a complete management of uncertainty— 
there are worse situations to come. Stochastic uncer¬ 
tainty makes the situation almost certain if the random 
variables involved are supplied with the respective 
probabilities. The situation is more evasive when the 
unknown factors cannot be statistically described. 
This may happen either when (i) a probability distri¬ 
bution for | exists basically, but cannot be evaluated 
by the moment of decision making, or when (ii) no 
distribution for | exists at all. 

An example of situation (i) may be a layout of a 
computer information system dedicated to servo ran¬ 
domly arrived demands. The probability character¬ 
istics of these arrivals would have been inferred sta¬ 
tistically if this system or similar had been in operation 


1 The method of probability theory, as applied to practice, 
is essentially''based on the assumption that low probability 
events are impossible (this topic and the principles underlying 
the assignment of significance levels a = 1 — P are discussed 
in [2]). 
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for a sufficiently long time. However, by the scheduled 
term of design no such information is available. How 
then will we manage the project decision making? 

Certainly, we can, upon some thought, intuitively 
assign some characteristics to the random factors | 
and optimize a solution x in this background, i.e. as 
above, in the mean, or by imposing stochastic con¬ 
straints. This treatment would, undoubtedly, be 
better, though not very much, than those with a deci¬ 
sion made at random. A more sound approach would 
be to set some decision variables x free to vary, then 
option a solution in full awareness of its being not 
the best one and implement it. As the implementation 
expertise piles up, we could consistently change the 
free decision variables striving to increase the effective¬ 
ness. The algorithms of this type, improving in the 
process of control, are labeled adaptive. Their advantage 
is in that they unburden us from the need to collect 
prior statistics, and are liable to readapt in response 
to changes in the environment. With more expertise, 
such an algorithm gradually improves similar to how 
the human improves on committed mistakes. 8 

Now we turn to the most difficult and unpleasant 
situation (ii), that of no probability characteristics for 
£, which, thus, cannot be deemed random in the ordi¬ 
nary sense. A little comment is here in order on the 
difference between uncertainty and randomness. As 
will be recalled, probability theory refers the term 
‘random event’ to the events which recur and, more 
important, have a property of statistical stability. 
The latter implies that similar trials with random out¬ 
come tend to assume a stable distribution upon mani- 


6 For a mathematically rigorous discussion on adaptive algor¬ 
ithms, we refer a mathematically mature reader to [31J. 
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fold repetition. The frequencies of the events tend then 
to the respective probabilities, and the sample means 
to the mathematical expectations. Tossing a coin 
many times, say, the frequency with which heads 
appear gradually stabilizes and ceases to be random. In 
weighing a body many times with a balance, the aver¬ 
age result ceases swinging and becomes more uniform. 
These are examples of stochastic uncertainty, “favor¬ 
able” in our terms. 

There is, however, nonstochastic uncertainty which 
we will loosely label “unfavorable” in reference to its 
analytical “amenability”. This time the factors \ are 
intangibles. They are as ever unknown beforehand, but 
additionally there is no point in speaking of, or trying 
to evaluate, their distributions or other probability 
characteristics. 

Let us illuminate it by an example. Assume that we 
plan a production-and-sales operation which depends 
in its success on what will be the most popular length 
of ladies skirts, g, in two years. A probability dis¬ 
tribution for | cannot be inferred principally of any 
statistical data. Even if we turn back to history and 
start accounting for the multiplicity of experiments 
(years) in which £ experienced variations, it would 
hardly help in our outlook. A probability distribution 
of | simply does not exist, by much the same reason 
as does not exist a set of homogeneous experiments 
on which the quantity could have a required stability. 
Hence, we face the situation of “unfavorable” uncer¬ 
tainty. 

However untractable mathematically this situation 
might seem, it would be unwise to give up all mathe¬ 
matical efforts, as even in these adverse circumstances 
a preliminary computation might be worthwhile. 

Let us trace this statement in more detail. Any 
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unfavorable uncertainty compels us to content with 
guesswork about | only. If we make a right use of it, 
i.e. choose more or less plausible values for |, then 
the problem under study casts into deterministic 
framework and may be handled by ordinary tech¬ 
niques—indeed attractive and easy seem deterministic 
situations now! 

Still, we have not enough reasons for joy. Assume 
that having consumed much effort and time (both our 
own and computer’s) we have arrived at some plausible 
|. Would then a decision made for these | be equally 
good for other |? As a rule, it would not. Therefore 
its value is highly limited, and a good practice at 
this stage will be to choose a tradeoff decision rather 
than optimal for some conditions. This decision though 
not optimal for, possibly, any conditions will be 
nevertheless applicable throughout their entire range. 

Whatever feasible theory of tradeoff does not exist 
so far, although some efforts in this direction are made 
in game theory and decision making (see Chapter 8). 
A common practice is to leave the option of a com¬ 
promise decision for the OR analyst. He would run 
the problem many times for various | and x to estimate 
with these preliminary computations the strong and 
weak points of each version and make an option. The 
procedure does not necessitate (though it is interesting 
at times) the knowledge of conditional optima for 
each set of conditions £. Calculus of variation tech¬ 
niques step back in this case. 

Another valuable aspect of preparatory analyses 
in the studies involving unfavorable uncertainty 
should be emphasized. At this preliminary stage the 
analyses help screen off the decisions x £ X that at 
any conditions £ are inferior to those retained and, 
hence, uncompetitive. It may substantially reduce 
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the set X, at times to a few alternatives amenable 
to consideration and assessment by the decision maker 
in search of a suitable tradeoff. 

In remedying the unfavorable uncertainty it is 
always worthwhile to stage competition of various 
approaches and viewpoints. Among the latter one 
singles out owing to its mathematical stand that, 
of utmost pessimism. It states essentially that the 
decision maker working under unfavorable uncertainty 
must always expect the worst possible outcome and 
make the decision yielding best effect in the worst 
conditions. If in these conditions we gain a payoff 
W — W, then we may guarantee that at any other 
conditions it will not be less (the principle of guaran¬ 
teed result). This approach is attractive in that it 
explicitly formulates the optimization problem and 
thus paves the way for correct mathematical techniqu¬ 
es. Yet it is justified far not always. Its major applica¬ 
tion area covers so-called conflict situations where the 
conditions are decided by an intelligent person (oppo¬ 
nent) who responds to each our decision in the worst 
way for us . 6 In more “neutral” situations the principle 
of guaranteed gain is not the only possible and can be 
considered along with the other alternatives. If employ¬ 
ed, however, the extreme position of this viewpoint 
should not be left out of sight since a decision it can 
produce can be only overcautioned and very careful, 
which may not always be sound. Imagine a military 
commander who makes all his decisions working from 
a hypothesis that his opponent is extraordinarily 
intelligent, shrewd and resourceful to counteract 
every time immediately and in a most unpleasant 


6 In more detail various approaches to choosing the decision 
alternatives under uncertainty are considered in Chapter 8. 
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way. Hardly would a strategy of this commander be 
a success. On the contrary, in any real situation, the 
effort of decision maker should be devoted to revealing 
where the weak points of the opponent lie so as to use 
the information in inventing a deceptive strategy. 

A spirit of pessimism is even less appropriate in the 
situations where the decision maker is not opposed by 
any enemy forces. The “pessimistic” computations 
should always be corrected by a sound amount of 
optimism. The other extreme viewpoint, that of blind 
optimism, might be as dangerous, yet some risk must 
be involved in any process of decision making. We 
should not overlook also that any decision made under 
unfavorable uncertainty is bound to be bad and there¬ 
fore hardly deserves to be substantiated with delicate 
and laborious computations. One should better think 
of where to collect the lacking information. Here all 
the ways are good if only they clarify the situation. 

In this connection we are bound to mention a rather 
original approach which, though not exciting lovely 
feelings in pure mathematicians, is useful and at times 
the solely possible. We speak of judgemental probability 
assessment. It is often employed in problems where 
forecasting is to be made under unfavourable uncer¬ 
tainty as, say, in futurology. The idea of the approach 
is roughly in the following. A team of experts is recruit¬ 
ed to answer a question, for example, to call the term 
of an invention, or assess the probability of an event. 
The collected answers undergo statistical treatment, 
as it were, and although the results retain the subjective 
nature, they do it in a lesser extent than if a single 
expert were interrogated. Similar assessments of prob¬ 
ability may be applied in studies under uncertainty. 
Each of the experts assesses the likelihood of various 
alternative conditions f assigning them some subjective 
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probabilities. Although each of the experts tends to 
introduce into the assessment his own bias, after 
averaging the result is more objective and useful (by 
the way, the estimates of real expertise disperse not 
so strongly as one might expect). In this manner 
a problem involving an unfavorable uncertainty re¬ 
duces, as it were, to an ordinary stochastic problem. The 
conclusions drawn in this section should, certainly, 
not be overemphasized; if added to other results, 
yielded by other approaches, they may, however, 
prove valuable in selection of an alternative . 7 

Finally we will make a general remark. Whatever we 
have done trying to substantiate a decision selected 
under uncertainty, an element of guess still remains. 
Therefore we ought not to make too higher a demand 
to thus produced decisions. Instead of pointing out 
a sole, explicitly optimal (from some viewpoint) 
decision, it would be a better approach to evaluate 
a region of feasible decisions which are if only insig¬ 
nificantly inferior to the others at whatever conditions 
(viewpoint) we consider them. Within this region we 
(the researchers) must recommend management to 
make its final choice. Displaying the decision making 
recommendations, the researcher must simultaneously 
convey the viewpoints on which these recommendations 
are based. 

2.3 Multiobjective Problems 

Inspite of substantial difficulties offered by uncertain¬ 
ty, the situations we have considered so far can be 


7 Judgement probability assessment is implemented in other 
arrangements as the simplest one outlined above with each 
expert expressing his opinion independently of the others; at 
times joint assessment is employed (brainstorming session). 

4-0432 
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regarded as most simple, since the objective or measure 
of effectiveness to be maximized (minimized) has been 
sole and obvious in each of them. Unfortunately, 
such problems are not a frequent practical occasion, 
they can be presumably met in dealing with small- 
scale and moderate-consequence ventures. In studies 
of large-scale complex actions which involve different 
interests of management and community, the perfor¬ 
mance cannot be as a rule completely described by 
a sole measure so that additional measures have 
to be designed. Owing to the diversity of objec¬ 
tives the problems of the type are termed multiobjcc- 
tive. 

As an example of such a problem consider a defence 
plan for an important facility to protect it from air 
attacks. We have at our disposal some antiaircraft 
weapons that must be rationally placed around the 
facility, arranged for mutual operation, have the 
targets distributed among them and adequate stock 
supply provided, and so forth. Assume that each of 
the enemy aircraft participating in a raid carries 
a powerful weapon which if hit home completely 
demolishes the facility. The primary objective of our 
operation is then to battle the aircraft off the facility 
to prevent any of them coming at an aiming point. 
Naturally a measure of effectiveness in this case will 
be built around the probability that none of the aircraft 
will break through our defence screen. Is it, however, 
the sole important objective? Certainly, not. The 
probability of rejection being equal, we would prefer 
a decision that cares for a larger count of enemy planes 
be shot down. Whence another measure, the average 
number of eliminated (hit) planes, which we also 
would like to maximize. Besides, we are not indifferent 
to our own losses—another measure we would like 
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to minimize. We might also wish that the amount of 
expended ammunition be minimal,-and so on. 

Another example, this time from the production 
planning field. Planning or reorganizing a production, 
management usually has to deal with several aspects. 
On the one hand, it would be desirable to maximize the 
output; on the other hand, the net profit must be 
made as large as possible. As for production costs, 
they must be least; while the labour productivity must 
be highest. If thought over a little more, the problem 
can produce additional objectives. 

The multitude of effectiveness measures, of which 
ones would like to be driven to a maximum, others 
to a minimum, is characteristic of any complex OR 
problem of value. We suggest the reader to exercise 
the acquired skill devising measures to assess the 
performance of a bus company. Consider which of the 
measures, from your point of view, is superior (most 
intimately connected with the objective); place the rest 
in the order of descending importance. This is a good 
example to convince that (a) none of the measures 
can be selected as the sole one, and (b) establishing a 
system of measures is not so simple a problem after all. 
Both the measures and their priority depend on whose 
interests are preferred when a decision is optimized. 

Resuming, we typically found a variety of objectives 
in a large-scale venture, with the associated quantita¬ 
tive performance indexes, W x , W 2 , • ■ ., W n , some 
of which are desired to be maximized, and the others 
minimized. 

The question now is whether there exists a decision 
such that can satisfy all these requirements. A frank 
answer to it is: no. A decision that maximizes one of 
the measures is as a rule unable either to minimize 
or to maximize the others. Therefore a formulation 


4* 
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such as ‘to produce maximal effect at minimal expen¬ 
ditures’ is nothing but a rhetorical phrase and must 
be discarded in any serious analysis. 

How would we manage then if we would have to 
assess the effectiveness of an action by several mea¬ 
sures? 

Nonexperts in operations research would tend to 
produce a single performance criterion for such a mul¬ 
tiobjective problem. They would devise a function 
of all the measures involved and consider it as a joint, 
generalized measure designed to optimize a sought 
solution. Often this measure is materialized as a frac¬ 
tion with the variables desired to be increased in the 
numerator, and those to be decreased in the denomina¬ 
tor. For instance, productivity and profit are above 
the line, and expense below. 

This approach cannot be recommended because it is 
based on an implicit assumption that a shortcoming 
of one of the measures may be made up at the expense 
of another; low output, say, at the expense of low cost. 
As a rule, this assumption does not hold true. 

Recall the person’s quality self-assessment criterion 
that was once jokingly devised by the famous Russian 
novelist Leo Tolstoy. He put it as a fraction with 
actual merits of a person in the numerator, and the 
self assessment in the denominator. At first sight, the 
criterion might seem logically solid. However, imagine 
a person almost devoid of merits and of nil opinion 
of himself. The criterion would judge him as one of 
infinitely high value which could hardly be agreed with. 

Similar paradoxical conclusions can not infrequently 
be drawn from a rational measure of effectiveness in 
which all the factors that contribute to advantage are 
put in the numerator, and those to disadvantage in the 
denominator. 
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Another, a bit more sophisticated generalized mea¬ 
sure of effectiveness in frequent use is drawn up as a 
weighted sum of individual measures each of which, 
Wi, enters with its own weighting coefficient ai reflec¬ 
ting the importance of a related measure: 

W = ai W t + a 2 W 2 + . . . (2.3-1) 

Those measures which are desired to be increased come 
in with positive weights, those to decrease with 
negative. 

For arbitrary weights a u a 2 , . . . this approach is 
nowhere better than the previous, if only the general¬ 
ized criterion this time does not go to infinity. Its 
advocates refer to the inherent analogy with that how 
a human working out a compromise mentally weighs 
all for and against assigning larger weights to more 
important factors. This similarity is true only in 
part since the weigting coefficients introducing various 
factors are not constant but vary accordingly as the 
situation alters. 

Illustrate the statement with an everyday life 
example. A clerk being out for the office considers his 
route to arrive in time. Which transportation will 
be most suitable? Streetcars go in short intervals, but 
run long; buses run faster, but arrive at longer inter¬ 
vals. He may, certainly, take a taxi, but this will 
be the most expensive way. Or another tack: cover 
a part of the distance by underground and then take 
a taxi. Yet there may happen to be no cars at the taxi- 
rank and then he will have to walk the last part of the 
distance, jeopardizing to be late more than if he took 
a bus. How will he behave? 

We face a typical (intentionally simplified) OR 
problem with two measures. The first is the mean 
expected time of being above schedule, T, which is 
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desired to be minimal. The second is the expected fare, 
F, which must also be kept at a minimum. The two 
requirements contradict each other, therefore, to 
make them compatible the clerk should arrive at a com¬ 
promise between them. He may be weighing sub¬ 
consciously all pros and cons employing somewhat of 
the generalized measure kind 

W = a x T + a 2 F =4- min (2.3-2) 

The trouble though, is that weighting coefficients a, 
and a 2 cannot be deemed constant. They depend on 
both the variables, T and F , and the situation. If, 
for example, the clerk has been reprimanded recently 
for being late, the coefficient for T will, likely, be 
increased, while the coefficient at F will, probably, 
be increasing away from a payday. If he assigns 
(as is usually done) a x and a 2 arbitrarily, then the 
resultant “optimal” decision will be as arbitrary in its 
essence. 

This example shows a typical trick for the situations 
of the sort—transfer of arbitrariness from one instance 
into another. A simple choice of a tradeoff decision by 
mental counterplaying all pros and cons for each 
alternative might have seemed arbitrary and not 
scientific enough, whereas manipulating a formula 
involving coefficients a x and a 2 (let alone their being 
assigned arbitrarily) looked more of a science. As 
a matter of fact, there is none of it in this manipulating 
either. 

We ought to abandon our hope for getting rid com¬ 
pletely of any subjective bias when choosing a decision 
alternative. Even in the simplest, one-measure prob¬ 
lems it will inevitably be present, emerging if only 
in the choice of measure of effectiveness or establishing 
a mathematical model. The bias is even more inevi- 
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table in a multiobjective problem. Seldom, though they 
might be, the situations turn up which make it clear 
immediately after evaluating the measures for all 
alternatives which of them to choose. If an alternative 
decision is superior to the others in all measures, then 
obviously, this one is to be preferred. More often 
though the studies are met in which the situation can¬ 
not be made as plain at a first attempt; the measures 
usually tend to divert in opposite directions. Therefore 
it would be worthwhile to conduct additional analysis, 
invoking, possibly, even the expressions of (2.3-1) 
type though treating their implications critically 
rather than without reserve. 

Helpless, as the considered situations might suggest 
it is, mathematical tooling is nevertheless of value in 
two aspects. First, it helps solving primal OR prob¬ 
lems, i.e. calculating for any decision x the relevant 
values of measures of effectiveness W x , . . ., 

however many they might count (incidentally, multi¬ 
objective nature is not a hindrance for primal pro¬ 
blems). Second and more important, it helps screen 
off those decisions of the set X of possible decisions 
which are inferior to their counterparts in all mea¬ 
sures. 

The procedure of screening is fairly simple. Consider 
a multiobjective problem of k measures, W lt W 2 , 
. . ., W h . To simplify the discussion, assume all these 
quantities are to be maximized (changing signs, as 
will be recalled, is enough to make the maximum of 
a minimum). Let in the set of possible solutions be two 
x A and x 2 such that all W u W 2 , . . ., W h for x x are 
equal or more than the respective criteria for the second 
solution, i.e. at least one of the measures assumes a 
larger value. Obviously then, there is no longer any 
sense to keep x i in X as it is dominated by x v We 
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reject x 2 as noncompetitive and go over comparing all 
the other decisions through all the measures. The set X 
may be drastically reduced as a result of this compare- 
reject procedure and retain only those (efficient) neither 
of which dominates the others. 
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To better illustrate how such decision can be evalu¬ 
ated, consider an example having two measures each 
of which is to be maximized. The set X consists of a 
finite number of possible alternatives, x 1 , x 2l . . ., x n . 
To each of them we may put in correspondence the 
relevant values of the measures W 1 and W 2 . We will 
plot these values as points at a IT, versus W 2 plane, 
bearing the numbers of the decisions (see Fig. 2.3-1). 

As we can observe, of all the set X we can refer to 
as efficient only the decisions x 2l x 5 , .r,„, and x xl bor¬ 
dering from right and top the possible options area 
(dashed line in Fig. 2.3-1). For any other decision, 
there exists at least one such that dominates it either 
in W x or W 2 or both. It is only for the decisions on the 
upper boundary that no dominating decision exists. 

Now that we have evaluated the efficient decisions 
out of the whole set of those possible, we may confine 
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our further search within the new “efficient” set. 
Figure 2.3-1 depicts it as consisting of four decisions, 
x 2 , x 5 , x 10 , and x lv Decision x u is seen to be the best 
in Wj, while x 2 the best in W 2 . It is up to the decision 
maker to choose the alternative that would satisfy 
him in both criteria. 

In a similar manner the efficient set is constructed 
when the measures are more than two (when more than 
three, a graphical representation can no longer be 
visualized, although geometrically a problem may be 
treated essentially along the same lines). The efficient 
set is easier to deal with than the set X. The final 
decision making is as ever left to management since 
what is required at this stage is to arrive at a trade-off 
decision which might be nonoptimal and nevertheless 
suit in several aspects; it will be made in an informal 
fashion by a decision maker bearing the responsibility 
for the final choice. 

The procedure of selecting an alternative solution, 
if repeated several times, can yet form a basis to work 
out certain intuitive rules (called heuristics) with 
which a computer may proceed arriving at a final 
solution by its own. This approach received accordingly 
a name of ‘heuristic programming’. It is essentially 
as follows. Assume that an expert or, still better, a 
group of experts having many times selected trade-off 
solution to a multiple alternative OR problem yields 
a series of estimates for various conditions a. With 
these estimates at hand and enough time one can, 
say, arrive at a reasonable choice of weights a r , a 2l . . . 
. . ., a n in expression (2.3-1), which generally depend 
on both a and measures W lt W 2 , . . ., W n proper. 
A generalized criterion built with the weights may 
then be conveyed to a computer authorizing it to 
choose a solution by itself. The heuristic programming 
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approach has at times to be pursued when there is 
virtually no time for thinking a compromise decision 
over as, say, in a combat operation. It is also employed 
when an automatic control system is established in 
full charge of a process or system. 

The computer also proves of value when the analyst 
has an ample time to come to conclusion by himself. 
He sits at the console and via the keyboard instructs 
the computer to calculate the values of measures 
W t , W 2 , . . ., W n . With these values on the display, 
he can critically assess the situation and appropriately 
correct the weighting coefficients, or other parameters 
of the control algorithm. 

An often applied strategy of reducing the number 
of objectives in a multiple objective problem relies on 
evaluating a major measure, say, W 1 , and driving it 
to a maximum, while imposing some constraints on the 
other measures, W 2 , . . ., W n , requiring that they be 
not lower than specified values w 2 , . . ., w n . For in¬ 
stance, in the optimization of production planning, 
management may require that the profit be maximized, 
items variety target be attained, and the costs of 
production be not above those specified. This approach 
translates all the measures, lest a major one, into the 
rank of given conditions a. A certain freedom in assign¬ 
ing the margins w 2 , . . ., w n still persists so that the 
analyst may introduce the relevant corrections inter¬ 
actively gauging the problem via a computer. 

Another way of arriving at a tradeoff solution is by 
means of what we would call successful concessions. 
Let the measures be ordered in decreasing priority. 
First, we seek a solution which maximizes the first 
(preemptive priority) measure so that W t — W*. 
Next, considering low accuracy of initial data and the 
relevant practical environment we may assign some 




2.3 Multiobjeclivc Problems 


59 


concession which we can afford in stepping \\\ 

back in order to maximize the second measure W 2 . 
With this limitation on requiring it be not lower 
than W* — A W lt we seek a solution which maximizes 
W 2 . The following step will obviously be an assign¬ 
ment of a concession in W 2 at the expense of which W 3 
can be maximized, and so forth. The advantage side 
of this approach is in that it immediately discloses 
what should be the amount of tradeoff in one measure 
to gain in another and how large will be the winning. 

Whatever stated, the multiple objective problem, 
being ill-structured, cannot be handled mathematically 
to a final point; the final decision is always left up to 
the decision maker. The analyst, or OR task group, 
supplies the manager with the data disclosing the ad¬ 
vantages and disadvantages of each alternative solution 
in order to facilitate his decision towards more objecti¬ 
vely substantiated preference. 

* * * 


To conclude, a few words are in order on the systems 
theory approach to decision making problems. 

Now that scales and complexities of operations in¬ 
crease dramatically, ever pressing become the prob¬ 
lems of optimal control concerned with large, dynam¬ 
ically interconnected systems involving a great 
number of units and subsystems organized into hierar- 
chial structures. The principle can be exemplified 
by a large corporation which includes relatively self- 
governed subsidiaries with their plants and factories 
which in turn can be subdivided into shops and facili¬ 
ties. Trying to optimize the performance of one element 
of such a complex system, the manager must not 
overlook the interconnections that exist between the 
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units at various levels of the hierarchial structure. 
The existing influence implies that a segment cannot 
be optimized in isolation from the entire system. 

Even within a smaller system, a plant, a sharp 
increase of production output as a result of an optimi¬ 
zation is only a half of a success. If the manufactured 
items pile up—better if on the shelves, worse if in the 
yard—because the transportation facilities are not 
ready to carry them out, the inevitable losses may 
offset the gain due to the optimized production boost. 

If we go further and suppose that the carriers were 
in abundance and there were no problems about trans¬ 
portation of the goods, however, the market stand 
had not been consulted so that the delivered surplus 
commodities swiftly diminished in price along with 
the other brought merchandise, it certainly would 
inflict losses—again resulted from “local” planning. 

What sort of planning should then be recommended? 
It certainly must not be an all-inclusive, tightly- 
composed brain work, governed hierarchially upside 
down and controlling all up to the smallest thing in 
a shop. This is neither possible, nor worth striving at. 
A sound control for a sophisticated hierarchy should 
be organized so that each command from an upper 
level downward be given in general terms providing 
some freedom for local initiative, though, the objec¬ 
tives must be formulated in such a way that working 
towards them a unit performs in accord with the 
interest of upper levels and the system as a whole. 

This is obviously easier said than done. A mathe¬ 
matical theory'of large hierarchially structured systems 
is now still under development. The effort is devoted 
to creating mathematical tools which would appro¬ 
priately describe such systems and decompose them 
into smaller subsystems or elements more suitable for 
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studies, yet so far efficient methods of control for 
such systems are still wanted. Practically, the ‘systems 
approach’ in the OR field is materialized as a treat¬ 
ment of a subsystem being optimized as a part of a 
larger system within which the subunit undergoes an 
evaluation of the influence it may impose on the per¬ 
formance of the whole system. 


Chapter 3 

LINEAR PROGRAMMING 


3.1 Linear Programming Problems 

In the previous chapters we considered mainly the 
methodology of operations research, studied various 
types of problems, evaluated ways to their solution, 
etc., while putting aside the subject of mathematical 
techniques needed for that purpose. This and ensuing 
chapters will shortly discuss some of these techniques 
widely employed in OR studies. We will not go into 
the details of their implementation, placing main 
emphasis on their basic structure. 

We have already mentioned the simplest problems 
in which the measure of effectiveness (objective func¬ 
tion), W , explicitly evolves from the orientation of 
the operation at hand and the conditions of the opera¬ 
tion are specified beforehand (deterministic problem). 
For this class of problems the objective function 
depends upon two sets of parameters only, namely, 
defined conditions a, and controllable (decision) vari¬ 
ables x: 

W = W (a, x) (3.1-1) 

It is important to recall that the specified conditions 
a contain the constraints imposed on the decision 
variables. 

Let a solution to our problem be a set of n decision 
variables x x , x 2 , ■ ■ •, x n , or an n-tuple, 
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It is required to find such values of x which maximize 
(or minimize), i.e. yield an optimum for, the objective 
function W. 

The problems seeking the values of parameters that 
optimize the function subject to the constraints im¬ 
posed on the arguments have come to bo known as 
problems of mathematical programming. 1 They differ 
in complexity which is decided by (i) the type of the 
function relating W with decision variables, (ii) the 
dimension of problem, i.e. the number of decision 
variables x x , . . ., x„, and (iii) the type and number of 
constraints imposed on the variables. 

The simplest (and best developed) problems of 
mathematical programming are the so-called linear pro¬ 
gramming ( LP ) problems. They are interesting in that 

(a) their measure of effectiveness (called mainly 
‘objective function’ for this class of problems) depends 
linearly on the decision variables x x , . . ., x n , and 

(b) the constraints imposed on the variables assume 
the form of linear equations or inequalities in x lt . . . 

• • • , ■ 

These are recurrent problems in practical applica¬ 
tions, for example in resource allocation, production 
planning, transportation scheduling, to name only a 
few. This is only natural, as in many practical prob¬ 
lems incomes and expenditures are linear functions 
of the amounts of acquired or utilized materials. For 
example, the total cost of a lot of goods varies in direct 
proportion with the number of items in the lot; trans¬ 
portation payments are again effected in direct pro¬ 
portion to carried weights—the examples may queue 
at length. 

1 The word programming does not refer to the programming 
of a computer; it means the programming or allocation of items 
or entities. 
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Of course, we are far from claiming all the practical 
relationships being linear; we would content ourselves 
with a modest statement that linear, or close, depen¬ 
dencies are frequent occasions—a valuable fact, too. 

To get a better sense of the matter, consider several 
LP problems. 

A blending (diet) problem. A feed mix for livestock 
at an animal farm is composed to meet some minimum 
daily calorie requirements of protein, carbohydrate, 
and fat. For simplicity assume that only four food¬ 
stuffs are available, F x , F 2 , F a , and F x . The cost of 
unit of each food is respectively c x , c 2 , c 3 , and c 4 . 
The calorie content of each food with respect to pro¬ 
tein, fat, and carbohydrate on per unit basis is shown 
in Table 3.1-1 by symbol a t j, where i stands for the 
number of food-stuff, hence i — 1, 2, 3, 4; and / stands 
for the number of nutrient: protein = 1 , carbohydrate 
= 2, fat = 3. 


Table 3.1-1 


Food 

Protein 

Carbohydrate 

Fat 

Cost 

Ft 

a n 

a 12 

a is 

Cl 


a 21 

a 22 

a 23 

C 2 

^3 

a 3l 

a 82 

fl 33 

C 8 


a 4 i 

a 42 

°43 

*4 


The problem is to blend a mix, that is, to assign an 
amount to each food-stuff component, such that the 
protein content of mix be not less than b x units, carbo¬ 
hydrates not less than b 2 units, fat not less than b 3 
units, and keep it at a minimum cost. 

Let us build a mathematical model. This time the 
procedure will be rather simple. Denote as x lt x 2 , x 3 , 



3.1 Linear Programming Problems 


65 


and * 4 the amounts of each food-stuff in the mix. The 
objective we are going to minimize is the cost of the 
mix, C, which linearly depends on the controllable 
variables x 1 , x 2 , x 3l and * 4 : 

C c x x x ~l~ r 2X2 ” 1 " ^3*^3 ~~h r 4 x 4 
or in a more concise form 

4 

C = Sei*i (3.1-2) 

i—i 

Thus, we have obtained the objective equation and 
it is linear. Next we write formulas for the constraints 
of protein, carbohydrate, and fat content. Considering 
that x x units of food F x contain a xl x x units of protein, 
x 2 units of food F 2 contain a 21 x 2 units of protein, and 
so on, we arrive at three inequalities 

^ 11^1 ’ ^' 21^2 ~b ^ 31 X 3 “H ^ 41^4 ^1 

a 12 x x -f a 22 x 2 + a 32 x 3 + a i2 x 4 > b 2 (3.1-3) 

a l 3 x l ~f" a 23 X 2 a 33 X 3 + <143*4 > b 3 

constraining the variations of decision variables x x , x 2 , 
* 3 , and * 4 . 

The problem reduces thereby to the following: 
find nonnegative values of decision variables 
*j, x 2 , * 3 , and * 4 such that satisfy the constraint 
inequalities (3.1-3) and simultaneously minimize 
the linear objective function of these variables: 

4 

C = c t Xi=>- min 

i=l 

What we have obtained is a typical problem of linear 
programming. We postpone the discussion of its solu¬ 
tion methods until later to illustrate three more prob¬ 
lems of the kind. 


5-0432 
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An inventory scheduling problem. A factory is set 
up to produce three products P u P 2 , and P 3 . The 
company that controls the factory has a contract to 
supply not less than b t items of P 1 , not less than b 2 
items of P 2 , and not less than b 3 items of P 3 . The 
factory may manufacture still higher numbers of the 
products, but the demand bounds them from above: 
not more, respectively, than p 4 , p 2 , and j} 3 items each. 
Each product requires for its manufacture four types 
of raw material m 1 , m 2 , m 3 , and m 4 . The stocks of 
each material are limited by amounts of y^, y 2 , y 3 , 
and y 4 units. To produce an item of product Pj the 
factory needs a t j units of raw material (i = 1, 2, 
3, 4). The first subscript, i, is seen to refer to the prod¬ 
uct, and the second, /, to the raw material. The values 
an are summarized in Table 3.1-2. 

Table 3.1-2 


Raw 


Products 


[materials 

Pi 

Pi 

Pa 

m 1 

a n 

a 21 

a Sl 

m 2 

“12 

a 22 

a 32 

m a 

a 13 

a 2S 

a S8 


fl 14 

a 24 

a S4 


The company realizes a profit of c ± on each item of 
P x , c 2 on each item of P 2 , and c 3 on each item of P 3 . 
How should the manufacturing of the products be 
arranged subject to the constraints placed by the 
contract and demand and so as to obtain the largest 
possible profit? 

Let us formulate the problem in the form of a linear 
program. The controllable variables will be x u x 2 , 




3.1 Linear Programming Problems 


67 


and x 3 , which are the numbers of items of P x , P 2 , 
and P 3 which the factory will produce. The necessity 
to meet the contract obligations will be presented by 
three constraint inequalities: 

x x > b x , x 2 > b 2 , x 3 > b 3 (3.1-4) 

Three more inequalities appear due to the restrictions 
of demand: 

*i < Pi, x 2 < p 2 , x s < p 3 (3.1-5) 

Besides, the factory must not run out of raw material 
stocks. Accordingly, for four types of material we get 
four constraint inequalities: 

^11*^1 “I” ^21 X 2 d~ ^-31 x 3 Yl 

a l2 x l “f" a 22 X 2 d~ a 32 X 3 ^ Yi 

®13®1 "l - a 23 X 2 “I" ®33 x 3 ^ Y3 

®14®1 ~f~ ®24 X 2 T" ^34 x 3 ^ Y4 

The profit realized on a production schedule 
(x x , x 2 , x 3 ) will be: 

L = c x x x + c 2 x 2 + c 3 x 3 (3.1-7) 

Thus we have obtained again an LP problem: find 
(choose) nonnegative values of variables x x , x 2 , and x 3 
such that satisfy the constraint inequalities (3.1-4), 
(3.1-5) and (3.1-6) and maximize the linear function 
of these variables: 

L = c x x x + c 2 x 2 + c 3 x 3 =$* max (3.1-8) 

This problem is closely akin to the previous one, the 
only difference being that this time constraint in¬ 
equalities are more in number and the objective is 
maximized rather than minimized (we already know, 
however, that the latter discrepancy is insignificant). 


6 * 
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The next problem prepares the constraints of still 
another type. 

A production scheduling problem. A weaving mill 
manufactures three grades of cloth, G x , G 2 , and G 3 , 
on two types of looms of which there are machines 
of type 1 and N 2 machines of type 2. The looms differ 
in production rate having various output for different 
cloths. The respective rates, a t j, are summarized in 
Table 3.1-3 where i refers to the type of machine and j 
to the grade of cloth. 


Table 3.1-3 




Cloth 


Loom . 




type 

Gl 

g 2 

G3 

1 

a ll 

a 12 

°1S 

2 

a 21 

a 22 

a 23 


The profit for each meter of cloth is c lt for a 
meter of G 2 is c 2 , and for a meter of G 3 is c 3 . 

To fill an order, the mill must produce monthly not 
less than b x , b 2 , and b 3 meters of each cloth, respectively. 
The warehouse capacity, however, demands that the 
produced amounts must not exceed, respectively, 
Pn p 2 , and p 3 meters. Besides, the operating conditions 
necessitate that all the machines be loaded. 

The problem is to assign the machines to the pro¬ 
duction order so as to realize the largest monthly 
profit. 

At first sight, what is formulated is a tween prob¬ 
lem to the previous one. The temptation is very 
strong to denote the amounts of cloths in a production 
schedule by x t , x 2 , and x 3 and maximize the total 
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profit c x x x + c 2 x 2 -f- c 3 x 3 . Yet we should stop here 
and ask ourselves: where are the capacities of the 
looms? Upon a little thought it will be obvious that 
this time the decision variables are the numbers of 
machines of type 1 and type 2 employed in the manu¬ 
facture of each cloth rather than the amounts of the 
cloths. For convenience we denote the decision vari¬ 
ables by the letter x with two variable indices, the 
first of which refers to a type of machine, and the second 
to a cloth. All in all, it yields six variables. 

X 11 X 12 x 13 (3.1-9) 

•^21 *^22 ^23 

where, say, x n stands for the number of type 1 ma¬ 
chines employed in the manufacture of grade cloth. 

We have come to formulating one more problem of 
linear programming. First, put down the constraints 
imposed on the decision variables x i; -. The major of 
them are the restrictions set by the order: 

^12*^12 ^22^22 ^2 (3.1“10) 

“I - ^23^23 ^3 

This done, come over to satisfy the warehouse limi¬ 
tations which yields three more inequalities: 

-k a 2l X 21 ^ Pi 

a l2 X l2 + a 22 ;r 22 < p 2 (3.1-11) 

^■13 X la ^23 X 23 ^ P 3 

Now is the turn for writing the conditions that all 
machines participate in the manufacture. Recall that 
the total number of the looms of type 1 employed in 
the manufacture must be N u and the machines of 
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type 2 amount to N 2 - Whence these two equalities: 

Xn + * 12 + 3:13 = Nl (3.1-12) 

%21 ”1“ X 22 ~4” ^23 “ N 2 

Finally, put down the total profit produced by all 
the cloths. The total amount of cloth G x manufactured, 
a n x n + a 2x a: 21 , produces the profit^ (a u x u + a 21 a; 2 j). 
The same type of reasoning yields, for a production 
schedule (3.1-9), the total profit of the mill for a 
month: 

L — c 1 (u n x IX “1“ a 2l x 2l) "4" ^2 ( a 12 x 12 “t - 0-22 x 22) 

c 3 (a 13 x 13 ^ 23 ^- 23 ) 

or in a concise form 

3 2 

£ = 2 c ]Yi a u Xij (3.1-13) 

}—l i—l 

This linear function of six arguments is to be maxi¬ 
mized, i.e. L =»■ max. 

In words, this linear program may be spelled thus: 
find nonnegative values of variables x lx , x 12 , . . ., x i3 
such that satisfy the constraint inequalities (3.1-10), 
(3.1-11), and equalities (3.1-12) and maximize the 
linear function of these decision variables (3.1-13). 
The total count of the problem constraints amounts to 
eight: six inequalities and two equations. 

A raw material allocation problem. A large corpo¬ 
ration has three production plants, P x , P 2 , and P s 
in demand each, respetively for a x , a 2 , and a 3 units 
of a raw material. Five sources variously distanced 
from the plants can supply this material at various 
transportation costs per unit tabulated in Table 3.1-4. 

The capacities of the suppliers are limited by the 
available stocks so that the sources S x , S 2 , S 3 , S t , 
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and S 5 can supply not more than 5j, b 2 , b 3 , 6 4 , and b 3 
units, respectively. It is required to design a ship¬ 
ment schedule, i.e. to allocate the amounts sup¬ 
plied by each source to each plant, such that the de¬ 
mands are satisfied at least possible transportation 
costs. 


Table 3.1-4 


Supplier 

1 'iani 







Si 

s 2 

S3 

S 4 

s 5 

Pi 

C 11 

C 12 

C 13 

C 14 

C 15 

Pz 

C 21 

c 22 

C 23 

C a4 

c 25 

^3 

c 3l 

c 82 

C 8S 

c 34 

e 85 


Formulate again the LP problem. Let x i} denote the 
amount of material received by the ith plant from the 
;th source. In total, a schedule will involve 15 decision 
variables: 


*11 

*12 

*13 

*14 

*15 

*21 

*22 

X 23 

*24 

*25 

*31 

*32 

*33 

*34 

*35 


(3.1-14) 


The constraints on demand must state that each 
plant receives as much material as it demands, 
whence 


*11 "p X 4 2 *13 "l” *14 *15 — 

*21 + *22 + x 23 + x 2i + x 25 = a 2 (3.1-15) 

*8i + *32 -f- 233 + x 34 -f- £ 35 = a 3 

Next put down the constraints evolving from the 
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capacities of the suppliers: 

*n + *21 + * 3 i < bj 

*12 4 ~ *22 4 ~ *32 ^2 

*13 + *23 + *33 < &3 (3.1-16) 

*14 + *24 + *34 < ^4 

*15 + *25 + *35 < b 5 

Finally come the total costs which we are to mini¬ 
mize. With reference to Table 3.1-4 we may put them as 

3 5 

L = 2 S c^Xj^min (3.1-17) 

i=l 7=1 

We again have obtained a linear program: find non¬ 
negative values of variables j such that minimize 
the linear function (3.1-17) subject to the constraints 
(3.1-15) and (3.1-16). 

Summing up, we have considered several typical 
LP problems. They have similar formulations and 
differ only in that in ones the objective must be maxi¬ 
mized whereas in others it must be minimized; in some 
problems the constraints are all inequalities, in 
others they may be both inequalities and equations. 
There are LP problems where all constraints are 
equalities. These differences, though, are of minor 
importance since constraint inequalities may be readily 
transformed into equations and vice versa, as will be 
demonstrated in the ensuing section. 

3.2 Moving to Algebraic Solution 

Any LP problem may be cast into a standard form 
which within the framework of this text will be called 
the general LP problem. 
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It is stated as follows: find nonnegative values of 
variables x lt x 2 , ■ ■ ., x n such that satisfy the 
constraint equations 

@11^1 “1~ ^12*^2 l - * • • 1 &\ n X n — bi 

^22*^2 


• ■ • ^ 2 n^n b 2 


(3.2-1) 


b m 


a ml' t 'l "t~ fl m2' I; 2 -f- ffl n 

and maximize the linear function of these variables: 


Tj ~i~ ^ 3 x 2 max 


(3.2-2) 


Let us demonstrate how it can be done. First, recall 
that the case when L must be minimized rather than 
maximized is easily handled by changing the sign of L 
and maximizing L' = — L. Second, any constraint 
inequalities may be converted to equations by intro¬ 
ducing additional variables, called ‘slack variables’ 
or simply ‘slacks’. We demonstrate the procedure by 
an example. 

Suppose that it is required to find nonnegative values 
of variables x 1 , x 2 , and x 3 such that satisfy the con¬ 
straint inequalities 

3^ -1- 2.r 2 — x 3 > 4 
x 1 — 2a: 2 + 3x 3 ^ 10 

and maximize the linear function of these variables: 

L = 4x 1 — x 2 + 2,z 3 =j» max (3.2-4) 

We start with providing inequalities (3.2-3) with 
zeroes on the right-hand side, namely, 

3x y -f- 2x 2 — x 3 — 4^0 
— x x + 2a: 2 — 3x 3 + 10 ^ 0 


(3.2-5) 
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Denote the left-hand sides in (3.2-5) by y 1 and y 2 . 

ifi = 3a:! + 2x 2 — x 3 — 4 
y 2 = — x x -f 2x 2 — 3x s +10 


The inequalities (3.2-5) are seen to secure the non¬ 
negativity of y x and y 2 . After this manipulating, the 
problem reformulates as follows. Find nonnegative 
values of variables x x , x 2 , x 3 , y x , and y 2 such that 
maximize the linear function of these variables sub¬ 
ject to the constraint equations (3.2-6). The absence 
of the slack variables y t and y 2 from L can be easily 
remedied by assuming that they enter it with zero 
coefficients. Thus we have arrived at the general LP 
problem. The transformation from the original in¬ 
equality constraint formulation has been effected at the 
expense of increasing the number of variables by two 
(equal to the number of the former inequalities). 

The reverse transformation is possible as well. As¬ 
sume that we face a general LP problem with the 
equality constraints (3.2-1). Assume further that only 
r ^ to of these to equations are linearly independent 2 . 
Any standard text on linear algebra (see, for example, 
[4]) proves that n variables x u x 2 , ■ . ., x n can be con¬ 
nected into at most n linearly independent equations, 
so in general r ^ n. Another fact from algebra is that 
a system of r independent equations in n variables 
x t , x 2 , . . ., x n can always be solved for some r vari¬ 
ables, termed basic variables , relative to the rest k 


1 Equations are referred to as linearly independent if none 
of them can be obtained from the others by multiplying them 
by scalars and summing up, i.e. none is the consequence of 
the others. 
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— n — r variables, termed nonbasic. The resulted 
nonbasic variables can assume any values without 
violating the conditions (3.2-1). Consequently, to 
convert the constraint equations (3.2-1) into in¬ 
equalities, it is sufficient to solve (3.2-1) for some r 
variables chosen as basic, or those in solution, expres¬ 
sing them via the nonbasic variables. Recalling that 
all the variables must be nonnegative, put down the 
nonnegativity conditions in the form of constraint 
inequalities. Then forget, as it were, about the basic 
variables to manipulate by the nonbasic ones amount¬ 
ing to k — n — r. The objective function, L, must 
also be expressed through the nonbasic variables only, 
by substituting for the basic ones the respective ex¬ 
pressions in nonbasic variables. As can be seen the 
translation from a general LP formulation to that 
with constraint inequalities reduces the number of 
variables by the number r of independent constraint 
equations making up the constraining set in the general 
LP problem. We will not illustrate the procedure with 
examples, leaving the possibility of check to the 
reader. 

Summing up, any LP problem can be cast into a 
general LP problem formulation. This book contains 
no detail treatment of its solution techniques. They 
are discussed at length in many special texts (con¬ 
sult, for example, [4, 5]) and numerous texts on 
operations research (e.g. [6, 7]). The subsequent section 
considers only general principles of how to evaluate 
whether or not a particular general LP problem is 
soluble and how its solution can be found. The relevant 
computational algorithms will not be treated; the 
interested reader may consult for them the mentioned 
texts, for example. 
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3.3 A Geometric Method of Solution 

Consider the general LP problem which is formulated 
as follows: find nonnegative values of variables 
x x , x 2 , . . ., x n such that maximize the linear function 
of these variables 


L = c x x x + c 2 x 2 -f . . 

• + c n x n =>- max 

(3.3-1) 

subject to the constraints 



+ u X 2 x 2 

+ O'ln't'Ti = ^1 


U 2 X X X T" o, 22 x 2 

a 2 n x n — b 2 

(3.3-2) 


”1" ^ran^n 


For simplicity we assume that all the 

conditions 


(3.3-2) are linearly independent, i.e. r = m. 3 

Any set of nonnegative values (x x , x 2 , . . ., x n ) satis¬ 
fying the conditions (3.3-2) is called a feasible solution 
of the LP problem. A feasible solution at which the 
linear objective function (3.3-1) is optimum is an 
optimal solution. Hence, the problem is to find an 
optimal solution. 

This problem, however, is not always soluble. It 
may happen so that the equations (3.3-2) are incon¬ 
sistent. On the other hand, they may turn out con¬ 
sistent, but solely outside the area of nonnegative 
solutions, i. e. no collection of numbers x x ^0, x 2 ^0, 
. . ., x n 0 exists such that meet the conditions 
(3.3-2). And, finally, feasible solutions may exist, 
but none of them be optimal because the function L 


3 If among the constraint equations (3.3-2) some are redun¬ 
dant, i.e. can be obtained from the others by linear transfor¬ 
mations, this reveals automatically in solving the problem 
(refer to [4, 6]). 
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is not bounded from above in the area of feasible 
solutions. All these “hazards” might be solely prepared 
by thought of, or unnaturally formulated problems, 
though occasionally insoluble LP problems may result 
from insubstantial planning, say incomplete account 
of available resources. 

To get a better insight into the principles underlying 
the general LP problem, we take up a geometrical 
approach. Let the number of structural constraints 
(equations) be two units smaller than the number of 
variables, i.e., n — m — k = 2. This particular case 
enables us to treat the problem graphically on 
plane. 

As will be recalled, m linearly independent equations 
(3.3-2) can always be solved for some m basic variables, 
expressing them via the other, nonbasic variables, 
counting n — m — 2 in the considered case. Assume 
that the nonbasic variables are x x and x 2 (if they prove 
to the contrary, we may renumerate the variables) 
and the others, x 3 , x 4 , . . ., x n , are basic variables. 
Solving (3.3-2) for the basic variables yields 

x 3 = a 3l x x + a 32 pc 2 + p 3 

x 4 = a 4X x x + a i2 x 2 + p 4 (3.3-3) 


x n = a nl x 1 + a n2 x 2 + 

We will illustrate a graphic solution method in the 
(x x , x 2 ) coordinates. Since the variables x x and x 2 
must be nonnegative, the area of the allowable values 
is the first quadrant, that is, above the axis Oaq, on 
which x 2 = 0, and to the right of the axis Ox 2 , on 
which x x — 0. Figure 3.3-1 shows it as the region 
within the shaded boundaries. 
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We come now to constructing on the plane x x Ox 2 
the region of feasible solutions, or convincing ourselves 



Fig. 3.3-1 


that it does not exist. The basic variables x 3 , x 4 , . . . 
. . ., x n must be nonnegative as well as satisfy the 
equations (3.3-3). Each of these equations restricts 
the area of feasibility. To prove, assume in the first 



Fig. 3.3-2 Fig. 3.3-3 

equation of (3.3-3) x 3 = 0; this results in the equation 
of straight line 

a 31 xi + ct 32 x 2 + P 3 = 0 
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On this line, x 3 = 0; above it x 3 > 0, below x 3 <C 0. 
We stripe the half-plane where x 3 > 0 (see Fig. 3.3-2). 
(To avoid unnecessary clutter, the shading of the 
region of feasibility is supressed to points near the 
boundary.) Now, the region of feasible solutions, or 
RFS for short, lies in the first quadrant, above the 
line x 3 = 0. Proceeding in exactly the same manner, 
we will graph all the other constraints (3.3-3). Bound¬ 
ary shading will show for each of them the half-plane 
where feasible solutions may be found. 

In all, we have drawn n lines: two axes of coordi¬ 
nates, 0:v u and Ox 3 , and n — 2 lines x 3 — 0, x 4 = 0, . . . 

. . ., x„ = 0. Each of them defines the feasible half¬ 
plane where feasible solution set must belong to. The 
part of the first quadrant which simultaneously belongs 
to all these half-planes is the sought feasible solution 
area. Figure 3.3-3 shows the graph with an existing 
region of feasibility, that is, the system (3.3-3) in this 
case has nonnegative solutions. Note in passing that 
these solutions are indefinitely many since any pair of 
nonbasic variables taken from the region of feasible 
solution satisfies the constraints along with the basic 
variables defined by it. 

It may so happen that the region of feasible solution 
does not exist as a consequence of the equations (3.3-3) 
being inconsistent in the positive quadrant. This 
situation is illustrated in Fig. 3.3-4 showing no area 
that would lie simultaneously to the desired side of 
all lines. This implies that the respective LP problem 
lias no solution. 

Suppose that a region of feasibility exists and we 
have constructed it. Next step is to find an optimal 
solution in this area. To this end, we approach the 
condition (3.3-1), L => max, graphically. Substitut¬ 
ing the equations (3.3-3) into (3.3-1) yields L in terms 
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of nonbasic variables x x and x 2 . After collecting similar 
terms, we have 

L = y l x l + Y 2^2 + Yo (3.3-4) 

where Yi and are certain coefficients, and Yo is a free 
term which L lacked originally, yet which might 



appear upon the transformation to variables x x and x 2 . 
We will, however, discard it immediately recalling 
that the linear homogeneous function 

L' = Yi^i + Y 2*^2 (3.3-5) 

achieves its maximum with the same values of x x and 
x 2 . 

Thus we ought to set out a graphical attack on U. 
First we set L' — 0 and draw the resulting line Yx^i 
+ y^x 2 = 0 on the plane x x Ox 2 . This line—we shall 
term it “level line”—passes, obviously, through the 
origin. If we assign to L' some values, the line will 
shift parallel to itself. Travelling in one direction it 
will increase L', in the other decrease L' in value. 
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Figure 3.3-5 indicates with arrows the direction of 
increase. This figure plots it upwards to the right, 



Fig. 3.3-5 Fig. 3.3-6 

but it might be to the contrary—the matter depends 
upon the coefficients and y 2 . Now we plot the level 
line and the region of feasible solution (RFS) in one 
graph (see Fig. 3.3-6). Let us mentally shift the level 
line parallel to itself in the direction of the arrows 
(where L' increases). Where will it reach a maximum? 
Obviously, it will be at point A, the extreme point of 
the RFS in the direction of increase. 4 The coordinates 
of this point (xf, x|) yield the optimum values fQr 
the nonbasic variables which when substituted into 
(3.3-3) give the optimal values for the rest (basic) 
variables, i.e. xj, xj, . . ., x*. An important thing to 
note is that V attains its maximum at one of the 
vertices of the feasibility polygonal region where at 
least two basic variables turn to zero. The value of 


4 If the arrows pointed otherwise (downward to the left) 
the extreme point of the RFS would be point B. 


6-0432 
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zero might he assumed by more basic variables, 
should more straight lines x t — 0 pass through the 
point A. 

Can there be a situation with a feasibility region 
existing and an optimal solution nonexisting? Yes, 



it can if the function L' (and consequently L, too) 
is not of bounded variation in the RFS. An example of 
such an abnormal situation with L' not bounded from 
above is depicted in Fig. 3.3-7 (this misfortune is out 
of place, though, if a problem is reasonably posed). 

Figure 3.3-6 graphs the problem for which an optimal 
solution exists and is unique. The case when an optimal 
solution exists but is not unique can be visualized 
graphically as, e.g. that in Fig. 3.3-8 illustrating a 
maximum of U being attained at the edge AB parallel 
to the level line, rather than at a single point. This 
situation can be met in practice, but we should not 
worry about it. Anyway, L' reaches its maximum at 
some of the feasible polygon vertices (whether A or B 
is insignificant), therefore the search for an optimal solu¬ 
tion may be performed among the vertices of the RFS. 
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Having considered graphically the LI* problem for 
which n — m — 2, wo can now issue the following 
statement: 

an optimal solution (if any) is always attained at 
one of the vertices of the region of feasible solu¬ 
tion, at which at least two of the variables 
aq, x 2 , . . x n are zero. 

Similar rule is proved to be valid for the case 
n — to = A: > 2 as well (we can, however, no longer 
work it out graphically as we cannot draw in more 
than three dimensions). We content ourselves with 
solely formulating this rule. 

An optimal solution of a linear programming prob¬ 
lem (if any) is attained at a set of values of 
variables x x , x 2 , . . x n such that turns at least k 
of these variables to zero, the others being non¬ 
negative. 

For k = 2, this set of values plots as a point on the 
plane, situated in one of the vertices of the feasible 
solution polygon. For k = 3, the set of feasible solu¬ 
tions is no longer a polygon, but a polyhedron; the 
optimal solution lies at one of its vertices. A graphical 
treatment cannot be effected for k > 3, but geometrical 
analogy retains its attractive convenience. As before, 
we will speak of a ‘feasible solution polyhedron’ in a 
/c-dimensional space, and an optimal solution (if any) 
will again be sought at one of the vertices of this 
polyhedron, where at least k variables are zero and 
the others nonnegative. To be concise, we shall term 
such a vertex a “basic point” and the relevant solution 
will be called a basic solution. 

This geometrical approach gives rise to the idea 
underlying most of the solution techniques in linear 
programming—that of iterative search. To demonstrate, 
solve the equations (3.3-2) for some of the to basic 

6 * 
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variables and write them in terms of the rest k non- 
basic variables. Set these nonbasic variables equal to 
zero. If we are fortunate enough, this will yield a 
basic point. Compute the basic variables at zero¬ 
valued nonbasic variables. If all of them turn out 
nonnegative, then we are lucky to get a feasible 
(basic) solution at a first trial and have only to optimize 
it. If they are not, then this choice of basic and non¬ 
basic variables fails to produce a feasible solution, 
i.e. the point lies beyond the RFS rather than on its 
boundary. A way out is to resolve the equations for 
another choice of basic variables in a non-arbitrary 
manner such that would get us closer to the feasibility 
region (for this purpose linear programming has devel¬ 
oped special techniques, but we will not concentrate 
on them in this text). 

Suppose that having repeated the procedure several 
times we have arrived at a basic solution. Half 
way there, but that’s still not all. We need to check 
whether or not this solution is optimal. Let us express 
the function L in terms of the latter set of nonbasic 
variables and then raise them from zero. If this only 
decreases the value of L , then the solution on hand is 
optimal. If not, then resolve the system of equations for 
other basic variables, again not in an arbitrary way 
but so as to near an optimal solution and not to leave 
the region of feasible solutions. For this purpose, too, 
linear programming has computational algorithms 
warranting that with each next resolving we will move 
closer to an optimal solution, or at least not away from 
it. We will not discuss these techniques here either. 
After a final number of such computational runs the 
objective will be attained and an optimal solution 
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found. In case an optimum does not exist, the imple¬ 
mented algorithm will demonstrate it. 

The reader may ask whether the entire matter is that 
simple. Why should we have been all those subtle 
solution techniques about when all we need is to 
barely examine one by one all the possible combina¬ 
tions of k nonbasic variables setting them equal to 
zero until finally an optimal solution is found? 

For the simple problems where the number of vari¬ 
ables is small, such an exhaustive search may indeed 
evaluate a solution and do this quickly enough. Appli- 
cational problems, however, not infrequently contain 
variables! and imposed constraints amounting in hun¬ 
dreds and even thousands. For these problems a simple 
exhaustive method becomes unfeasible because the 
number of basic and nonbasic variable combinations 
rises too high. To illustrate, at n = 30 and m — 10 
the number of possible combinations which yield non¬ 
basic variables with basic variables is 30 C 10 — 

30 045 015, that is' more than 30 million' combina¬ 
tions! The problem though is not of a sophisticated 
variety. 

The computational techniques developed for linear 
programming—the simplex method, the dual simplex 
method, and others, refer, for example, to [4, 7, 8]— 
seek for an optimal solution in direct iterative, rather 
than exhaustive, manner, appearing close to the 
solution with every step. Computer hardware manu¬ 
facturers supply as a rule the relevant linear program¬ 
ming software so that a person facing an LP problem 
need not learn manual programming which for the 
problems of this class might be extremely monotonous 
and time consuming. 




86 


Ch. 3 Linear Programming 


3.4 The Transportation Problem 

The previous section has been concentrated on some 
general approaches to linear programming solution. 
Among the discussed family of problems, however, 
there are some which owing to their outstanding 
structure are amenable to more simple solution meth¬ 
ods. We consider one of them, the transportation 
problem. The transportation problem requires the allo¬ 
cation of units located at a number of origins to a 
number of destinations in such a way that the alloca¬ 
tion is an optimum (least cost or maximum profit). 
It is formulated as follows. The amounts of units 
available at each of m origins (shipping sources) A ; 
(i = 1, . . ., m) are «j, a 2 , . . ., a m units, respectively. 
The amounts required at each of n destinations Bj 
are fq, b 2 , . . ., b„ units, respectively. These values are 
often referred to as rim requirements. The sums of all 
demands and supplies must be equal: 

m n 

2« ; =2&, (3.4-1) 

i=l j=\ 

The cost of transporting (or profit of allocating) one 
unit from origin A ; to destination Bj (i = 1,2,. . ., m; 
7 = 1,2,..., n) is Cij. All the numbers c i} are known; 
composed in a matrix form they may be written as 

C 11 c 12 • • • c \n 

C 21 C 22 • • • c 2 n (3-4-2) 


c m 1 Cm 2 • • * C mn 

For shortness, we will designate this matrix by the 
symbol [c^]. 
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The transportation costs are deemed to vary in direct 
proportion to the number of transported units. 

It is required to design a transportation schedule 
(i.e. to allocate the origins, destinations and transport¬ 
ed amounts) such that all the demands are met at a 
minimum cost. 

Transforming the problem into the framework of the 
LP problem, denote by xij the amount allocated from 
origin At to destination Bj. The values x X j must be 
nonnegative. Together they form the matrix 


X n x l2 ■ . 

• *^1 n 


X 2 l ^22 • • 

• *2 n 

(3.4-3) 

1 2 . . 

• %mn 



which we will shortly denote by [xij]. We will refer to 
a set of values [xtj] as a transportation schedule, and to 
the variables xij themselves as allocations or ship¬ 
ments. These nonnegative variables must satisfy the 
following constraints. 

1. The total amount of supply allocated from each 
origin to all destinations must be equal to the amount 
available at the particular origin. This restriction 
gives m constraint equations: 


■Til + #12 + • • • + T ln — a x 

X%i -f- X22 + • • • + n ~ 


Tml "I" “t - • • • T~ %mn — a m 


(3.4-4) 


2. The total amount of supply delivered to each 
destination from all origins must be equal to the 
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particular demand. This yields n constraint equations: 

^11 + X 21 + • • • + Xm 1 = &1 

*12 "H *22 “t" • • • *m2 ~ ^2 /0 / c\ 


*ln “1“ * 2 n “1“ • • • "l” *mn b n 

3. The total transportation cost of all allocations, 
i.e. the sum of xi } multiplied by the respective costs 
ctj must be minimal 

m n 

i = S S c u x ij = ^ m in (3.4-6) 

i=l 1=1 

where the double summation operator implies that the 
summation is performed over all combinations of 
changing indices i and 7 , that is, through all the 
origin/destination pairs. 

We have stated a linear program with constraint 
equations (3.4-4) and (3.4-5) and linear objective 
function (3.4-6) which is to be minimized. It is out¬ 
standing in that all the coefficients in (3.4-4) and 
(3.4-5) are unities, the feature that lends the problem 
amenable to very simple solution methods. We will 
discuss some of them in what follows. 

First of all notice that the constraints (3.4-4) and 
(3.4-5) are no longer linearly independent as their 
right-hand sides are related by the condition (3.4-1). 
The number of those linearly independent among 
equations (3.4-4) and (3.4-5) is therefore m + n — 1 
rather than the total number of equations m + n. 
The total number of variables Xjj in our problem is 
m X n\ whatever basic variables the equations (3.4-4) 
and (3.4-5) were solved for, their number would be 
exactly m -f n — 1 , while the number of nonbasic 
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variables 

k = mn — (m -f n — 1) = (m — 1) (n — 1) 

As will be recalled, an optimal solution to an LP 
problem is attained at a vertex of the feasibility area, 
where at least k variables assume the value of zero. 
For the case in question, an optimal shipment schedule 
requires that at least (m — 1) (n — 1) shipments must 
vanish, that is, no shipment is effected from respective 
origins to destinations. 

A shipment schedule will be termed feasible if it 
satisfies the constraints (3.4-4) and (3.4-5), i.e. all 
demands are satisfied and all supplies are exhausted. 
A feasible shipment schedule will be referred to as 
basic feasible schedule (solution) if it has at most 
m -| n — 1 positive shipments (the other shipments 
are zeroes), otherwise it is degenerate. A schedule [xij] 
will be termed optimal if among all feasible sched¬ 
ules it results in the lowest total transportation 
costs. 

Owing to the special structure of the transportation 
problem its solution requires no lengthy recursive 
manipulations of the systems of equations. Instead, 
the search for optimal solution is performed within 
a table containing the data of the problem set up in 
a certain order. The entries to the table are origins 
and destinations, demands and supplies, transportation 
costs, and, if allocated, shipments positioned in the 
respective cells. A transportation table has m rows 
and n columns. In the right-hand corner of each cell 
we place the transportation cost Ctj payed to transport 
a unit of commodity from At to Bj. A shipment will 
be placed in the center of the cell. The cell correspond¬ 
ing to the shipment from A; to Bj will be shortly 
denoted as (i, /) shipping route. 
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An example of transportation table with m = 4 and 
n = 5 is given in Table 3.4-1 where the data of the 
problem has been entered but allocations have not 
been made. 


Table 3.4-1 


Origin 

Destination 

Sup¬ 

ply 

Si 


S3 

B i 

Bs 

| 

Ay 


13 


7 


| 14 


7 


5 

30 











a 2 

1 

11 

1 

8 l 

I 

12 

1 

6 


8 

48 

1 


1 


1 


1 


1 


^8 

1 

6 

1 

10 

1 

10 

1 

8 

1 

| 11 

20 

1 


1 


1 


1 


I 


A 4 


14 


8 


10 


1 10 


1 15 

30 











De¬ 

mand 

18 

1 

27 

42 

15 

'26 

128 


First of all we should compose an initial (basic 
feasible) schedule. It may be readily effected by the 
northwest-corner method 5 . We will demonstrate it by 

6 Also referred to as the northwest-comer rale within the 
stepping-stone method with inspections. The rule states that 
the quantities shipped must begin in the upper left-hand corner 
of the table used in the method. — Translator's note. 
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means of the data in Table 3.4-1. The filling of the 
transportation table begins with the upper left-hand 
(northwest) corner, by satisfying the demand of desti¬ 
nation B v We allocate the required 18 units from the 
supply of A v The remaining 30 — 18 = 12 units 
after this shipment we allocate to destination B 2 . 
Since its demand is not satisfied in full, we allocate 
the completing 15 units from the stocks of A 2 . Using 
this procedure the table is filled with shipments xtj 
down to the lower right cell (see Table 3.4-2). The 
feasibility check for this schedule reveals that the rim 
requirements are satisfied, i.e. the sum of all shipments 
in a row is exactly the amount available at the respec¬ 
tive origin, and the sum of allocations in a column 
equals the requirement of the respective destination. 
It means that all the demands are met and all the 
supplies are exhausted (the sum of supplies is equal 
to the total demand, 128 units entered in the lower 
right cell). 

Henceforth we will fill into the cells only those 
shipments which are nonzero; the other cells are left 
unfilled. Let us verify now whether or not this initial 
solution is degenerate, i.e. whether the filled cells 
(with amounts to be shipped) are not too many. The 
number of unfilled cells in Table 3.4-2 is 12 which 
is exactly the figure (m — 1) (re — 1) = 12 to qualify 
as nondegenerate. 6 

Now is the time to test this schedule for optimality, 
i.e. to test whether its total transportation costs are 
minimal. It most likely will be not, since designing 
the transportation schedule we did not pay attention 

8 Another formula to test for degeneracy is m + n — 1. 
i.e. the number of fdled cells must be the rim requirements 
(rows and columns) minus one for the solution to be not degener¬ 
ate.— Translator's note. 
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Table 3.4-2 


Origin 

Destination 

Sup- 

Bl 

B* 

Bs 

b 4 

b 6 

ply 

^1 

18 

13 

12 

7 

■ 

14 

1 

B 

1 

5 

30 




■ 


■ 

■ 

» 

B 


a 2 


11 

15 

8 

33 

12 

■ 

■ 

1 

8 

48 






■ 

■ 

■ 



^3 


6 

■ 

10 


10 

11 

8 

1 

11 

20 


■ 


■ 


9 



■ 



A, 


14 


8 


10 


10 

26 

15 

30 








4 




De¬ 

mand 

18 

27 

! 42 

15 

26 

128 


to costs. The nonoptimality of this schedule can be 
evaluated easily. For example, it can be readily seen 
that it can be improved by having the lowest transpor¬ 
tation costs associated with the filled cells, which im¬ 
plies moving the columns around. For example, we 
may cut the shipment in route (2,3) of high cost, 
but instead raise the allocation of low cost shipping 
route (2,4). To secure nondegeneracy in this move¬ 
ment from used into unused cells we have to empty one 
of the filled cells. How many units may we move through 
the closed system of routes (2,4) (3,4) ->■ (3,3) -> 
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->(2,3), increasing shipments in the odd cell routes 
of the set and decreasing in the even? Obviously, not 
more than 11 units, or else the shipping in cell (3,4) 
would be negative. As can be verified, the movement 
of allocations does not violate the rim requirements— 
the supplies and demands are balanced. Performing 
this shift yields an improved transportation schedule 
displayed in Table 3.4-3. 

Table 3.4-3 


Origin ' 

Destination 

Sup- 

Bi 

[B t 

B 3 

B t 

B& 

ply 

A 

18 

13 

12 ^ 

7 1 

1 

14 

■ 


■ 

5 

30 




H 


■ 

m 

■ 



^2 

1 

11 



22 

12 

11 

6 

■ 

8 

48 


■ 






| | 


^3 

■ 

6 

■ 

10 



1 

8 

■ 

11 

20 


■ 

■ 

1 

■ 


■ 


1 



A* 


14 


8 


10 


10 

B 


3> 


1 


■ 


■ 

■ 

4 




De¬ 

mand 

18 

27 

42 

15 

26 

128 


Consider now what are the cost savings with this 
new schedule. The total shipping costs of the schedule 
presented in Table 3.4-2 are L t = 18 x 13 + 12 x 7+ 
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+ 15 X 8 + 33 X 12 | ( J x 10 | 11 x 8 + 4 x 

10 + 26 X 15 — 1442 whereas lliose in Table 3.4-3 
amount to L 2 = 18 X 13 + 12 X 7 + 15 X 8 + 
22 X 12 + 11 X 6 + 20 x 10 + 4 X 10 + 26 x 15 
=1398. 

Thus we have lowered the total shipping costs by 
1442 — 1398 = 44 monetary units. The result, though, 
may have been predicted with no resort to the total 
shipping costs account. To prove, the algebraic sum 
of shipping costs of the moving route cells taken with 
a plus sign if the shipment of the cell increases and 
with a minus sign if decreases—called the net cost— 
amounts for the above example to 6 — 8+10 — 
12 = —4. It means that in moving one unit through 
a closed set of routes (the so-called evaluation method) 
the shipping costs diminish by four. We have moved 

11 units which means that the costs must have been 
diminished by 11 X 4 = 44 monetary units, which is 
obviously the case. 

We now may conclude that essentially all we need to 
optimize shipping schedule is to transfer shipments 
round closed systems of routes having a negative net 
cost. 

Linear programming has the following statement 
proved for a nondegenerate shipping schedule. 

For each unfilled cell in a transportation table 
there exists a unique closed set of routes with one 
turning point in this cell and the others in filled 
cells. 

Therefore, looking for cost saving paths of negative 
net cost we must evaluate those unused cells with low 
shipping costs. If such a cell is available, it should be 
included in an appropriately chosen closed set of 
routes. The next step is to evaluate the net cost of 
moving a unit through this system and if it turns out 
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negative, then we should move through the path as 
many units as possible without driving any shippings 
onto a negative level. The unused cell, consequently, 
becomes filled and one of the used cells becomes empty. 
This is obviously equivalent to resolving the system 
of equations for other basic variables with the 
difference that the tabular method performs it far 
easier. 

Let us make another attempt at improving the 
transportation schedule given in Table 3.4-3. Suppose 
that we have chosen an attractive cheap route cell 
(1,5) worth 5. An obvious trend would be to raise the 
shipment in this cell at the expense of cutting ship¬ 
ments in some others (some routes would also have to 
ship more). Let us give an attentive treatment to 
Table 3.4-3 to evaluate a closed path whose first cell 
is (1,5) which is unfilled, and the other routes are 
used cells. Upon a little thought we arrive at the 
path: (1,5) -> (4,5) -*-(4,4) (2,4) -*(2,2) (1,2); com¬ 

pleting it we again move to (1,5). The odd routes of 
the path are marked with a plus sign implying that the 
respective shippings increase, the even routes are 
marked with a minus sign indicating that they will 
ship less. The entire path is graphed in dashed lines 
in Table 3.4-4. 

Compute the net cost of this path: 5 — 15 + 10 — 
— 6 + 8 — 7 — —5. Since the cost is negative, a 
movement through this path is cost-saving. Evaluate 
now how many units may we move through it. This 
is defined by the lowest shipment at a negative route 
cell of the path. This time the figure is again 11 (by 
pure coincidence). Multiplying it by the net cost of 
the path, which is —5, yields that we have saved 
55 monetary units more (the reader is invited to verify 
it by himself). 
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Evaluating in this manner unfilled cells included in 
the paths of negative net cost and moving through these 

Table 3.4-4 


Origin 

Destination 

Supply 

B, 

b 2 

B s 

B, 

Bs 

A, 

t8 13 

IZ [_7_ 

f-T- 

k 

k 

k 

30 

Az 

11 

WdL 
&— 

22 k 

7k 

1 

k 

48 

A 3 

6 

k 

k 

20 

, 

1 k 

i 
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k 

20 

A* 

14 

8 

k 

!*k 

1 

-<!>. 

k 

26 

30 

Required 

18 

27 

42 

15 

26 

128 


paths as many units as possible, we will lower the 
total transportation costs even more. However, we 
may not do this without end as we may not diminish 
it beyond zero, hence sooner or later we would arrive 
at an optimal schedule. This schedule will no longer 
contain an unfilled cell with a negative net cost of a 
closed path. This feature will signal us that an optimal 
solution is found. 

Linear programming has the solution methods that 
automatically evaluate unfilled cells of negative net 
cost, but we shall not pause on them as well as on 
other approaches to the transportation problem, which 
the reader can find in special texts, e.g. [5, 7, 8]. 
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In conclusion, we say a few words on the situation 
when the sum of all demands is not equal to the 
available supplies, i.e. 


2 «i^2 bj (3.4-7) 

i=l 3-1 

When 2 i=i «i > 2 Ai bj (the supplies are in 
excess of the requirements) ail the demands can be 
satisfied, but not all the supplies will be exhausted. 

This situation may be handled by introducing a 

fictitious destination, B ( , to absorb the difference 
(slack) 

in n 

6f = S <*i - 2 b j (3-4-8) 

3=1 3=1 

What should then be the costs of shipment from 
origins At to the fictitious location? Naturally, they 
should be assigned zero values since in reality nothing 
will be shipped to B t . Hence, for all origins ct t — 0. 

We augment the transportation table with a dummy 
column corresponding to destination B t and contain¬ 
ing zero transportation costs. Solution is then attemp¬ 
ted in the usual manner to result in an optimal trans¬ 
portation schedule 

Xu X 12 ... x ln Xit 

•^21 *^22 • • • *^ 2/1 *^2 f 


^ml %m 2 • • • %mf 

We should keep in mind, however, that the slacks 
Xi t are shipped nowhere: they remain at the respective 
origins A 


7-0432 
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The opposite situation with 2 i=i a i < 2 j=i bj 
(not enough stocks available to meet all the demands) 
may also occur. This difficulty may be alleviated by 
cutting the required amount in some way. If it is 
insignificant how rightly the supplies are allocated, 
but the sole objective is to ship them at a least pos¬ 
sible cost (no matter to what destinations they will be 
assigned), then we may introduce a dummy origin 
to supply the difference 2 j=i bj — 2 ;=i a i- We are 
not going to treat the topic in more detail, referring 
the reader to [6]. 


3.5 Integer Programming. 

The Concepts of Nonlinear Programming 

A remarkable number of practical problems which 
have similar formulations to LP problems differ from 
them in that the sought values of the variables are 
stipulated to be integers (whole numbers). These prob¬ 
lems are referred to as integer programming (IP) 
problems. The integer requirement substantially com¬ 
plicates their solution. 

Consider an illustration of such a problem. Suppose 
that we are to evacuate n precious items of art from 
a dangerous area. Their costs Cj together with their 
weights qi are known. The number of items which we 
could evacuate is limited by the capacity of the 
available transportation facilities, Q. It is required to 
set up a most precious (highest priced) choice of the 
collection such as could be carried away in Q. 

For the sake of consideration we introduce the so- 
called 0-1 integer variables xt (i = 1, 2, . . ., n) 
such that Xi — 1 if we select the ith item and xi — 0 
if not. Then, for a selected lot, i.e. with assigned values 
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to xi , the weight will be q x x r + q 2 x 2 + • • • + q n Xn- 
The restriction imposed by the carrying capacity is 

01*1 + 32*2 + . . . + q n x n < Q (3.5-1) 

The total value of the items, which we wish to 
maximize, is 

n 

L — 2j c i%i => max (3.5-2) 

i =1 

On the surface this problem formulation almost does 
not differ from that of a common LP problem: find non¬ 
negative values of variables x lt x 2 , . . ., x n such that 
maximize the linear function of these variables (3.5-2) 
subject to the constraint (3.5-1). At first sight, it 
might seem to be handled in the same way as a LP 
problem, by solely augmenting it with the constraints 
accounting for the solitary property of each item: 

^ 1, x 2 ^ 1, . . ., x n sg; 1 

Yet this addition does not go. A solution obtained 
in this way might appear fractional rather than in¬ 
teger and hence infeasible (can’t we load a half of a 
sculpture or a third of a painting 1). The considered 
problem is other than a LP, it is an integer program. 

A second thought that comes to mind is whether the 
problem could be approached by solving its linear 
program part and then rounding off the obtained values 
either up or down to the next integer 0 or 1. Unfor¬ 
tunately, this approach is also invalid. A solution so 
received might even not meet the constraint (3.5-1), 
that is, it might violate the imposed weight limitation. 
Even if it obeys the restriction, it might lie far from 
an optimum. Some problems, though, admit this 
rounding approach. For example, if the production 
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scheduling problem oi' Section 3.1 would have a solu¬ 
tion: 185.3 machines of first type must be assigned to 
production of first grade cloth, then we would simply 
round the figure to 185. If the point is to assign resources 
having the property of integrity (especially valuable 
or unique), the problem must be stated as an integer 
program. 

The integer programming problems in general re¬ 
quire a considerably more sophisticated approach for 
their solution than correspondingly sized conventional 
LP problems. Several solution techniques are known 
to be implemented for such problems; all of them, at 
whatever significant number of variables, are very 
complicated and laborious. We are not intended to 
cover these techniques here and refer those interested 
to the texts where more space is devoted to the topic 
(for example, [7, 8]). 

The nonlinear character of the integer requirement is 
subtle enough to accommodate an unexpected variety 
of other nonlinearities. A few words therefore would 
be in order on nonlinear programming. 

A general formulation of the nonlinear programming 
problem is as follows. Find nonnegative values of 
variables aq, aq, . . ., x n such that maximize the non¬ 
linear objective function of these variables 

W = W (aq, aq, . . ., x n ) =>■ max (3.5-3) 
subject to the constraints of arbitrary type, say, 
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No universally powerful general algorithm to solve 
nonlinear programming problems is known so far; 
each particular problem is approached depending on 
the form of nonlinearity of the objective function and 
the type of imposed constraints. 

Nonlinear problems are frequent occasions in appli- 
cational arenas; they take place, for instance, where 
the costs vary not in direct proportion with the amount 
of acquired or produced commodities (say because of 
discount rising with a purchased lot). Many of them 
can be approximated by linear models (linearized), 
at least in an area close to an optimal solution. If the 
linearization cannot be succeeded, still problems of 
applicational value are normally pursued towards 
models with the nonlinearity that can be safely handled 
by some known algorithms. A particular recurrent 
type involves quadratic programming problems whose 
objective function is a second degree polynomial in 
variables x u x 2 , . . ., x„, and the constraints (3.5-4) 
are linear inequalities (see [7, 81). 

A whole family of nonlinear programming problems 
can be solved to advantage by penalty function algo¬ 
rithms which reduce the constrained source problem 
to a sequence of “unconstrained” relaxed models the 
solutions of which approach a solution of the original 
problem. This is done by combining the constraints 
with the objective function in such a way that minimiz- 
iug the combined ‘penalty function’ penalizes con¬ 
straint violation. The basic idea may be exemplified 
as follows. Instead of imposing a strict constraint of 
the form / (x,, ar 2 , . . x n ) ^ 0, it may be added to the 
objective function W (x lt x 2 , . . ., x n ) in a form of 
penalty, say af (x x , x 2 , . . ., x n ), which would tend to 
prevent violation of the original constraint; here a 
is the penalty coefficient (negative when the objective 
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function is maximized, and positive when minimized). 
Varying a in absolute value one may gauge what 
variation it entails for an optimal solution 
(x*, x\, . . ., x%) to hold one when the variations prac¬ 
tically cease. 

Another method which has proved of value in ap¬ 
proaching nonlinear programs is random search tech¬ 
nique, which, as the name implies, is the random (with 
a play of chance) investigation of the solution space 
for the optimum. This method as well as the mentioned 
above and others the reader can find (if needed) in 
[7-91. 

Last but not least, a short mention is due for sto¬ 
chastic programming problems 7 . They arise in applica¬ 
tions where optimal solutions have to be sought in the 
conditions of incomplete certainty. For example, some 
variables in the objective function and the constraints 
vary at random, i.e. can he realized only as probability 
statements. This programming is also referred to as 
chance-constrained programming. Some of these prob¬ 
lems can be converted to ordinary deterministic 
models. This is the case, for example, when optimiz¬ 
ation is effected in the mean, the objective function 
linearly depends on the decision variables, and the 
only elements of randomness are coefficients of vari¬ 
ables in the objective function. With this formulation, 
the solution can he optimized having substituted for 
the coefficients their mathematical expectations (the 
mathematical expectation of a linear function, as 
will be recalled, is equal to the linear function of the 
expectations of the arguments). More involved, how¬ 
ever, are the situations where the imposed constraints 


7 Stochastic programming is a former misleading name of 
dynamic programming, too.— Translator'* note , 
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are random functions. These and other problems of 
chance-constrained programming are extensively cov¬ 
ered in [31]. 

In contrast to linear programming whose solution 
methods are well developed, settled, and never pose 
essential difficulties, the integer, nonlinear, and 
chance-constrained programming problems fall within 
the province of hard computational problems. In 
tackling them a resort is often made to approximate, 
heuristic optimization techniques, since analytical 
methods appear too laborious to handle even on a 
large computer. Not infrequently the efforts applied 
to gain an optimal solution entirely offset the reward 
that could be earned from this solution so that the 
game proves not worth the candle. This points out 
again to the necessity of applying a systematic approach 
to OR problems, which would account for not only 
an immediate winning in the operation at hand, but 
also the costs of pertinent optimization. 

The soothing fact is that ways are still being devised 
of improving existing algorithms. This, coupled with 
the increasing efficiency of package programs as well 
as computer hardware, is constantly making it eco¬ 
nomically possible to solve larger models. On the 
other hand, the size of optimization models parallels 
the trend in computational capabilities. One should 
also not overlook that the decision made in choosing 
an optimization algorithm is an important stage, at 
times even the most critical. 




Chapter 4 

DYNAMIC PROGRAMMING 


4.1 Concepts of Dynamic Programming 

Dynamic programming (DP) is an optimization tech¬ 
nique suitably adapted to work with the processes 
requiring sequential decision making to search for an 
optimum. These are known as sequential decision pro¬ 
cesses. A sequential decision process is loosely defined 
as an activity entailing a sequence of decisions taken 
toward some common goal. 

To illustrate, imagine a process that breaks down 
into a series of sequential steps or stages, for example, 
activity of an industry for several fiscal years, or 
penetration by a group of airplanes several echelons 
of antiaircraft defence, or a sequence of tests established 
to control the quality of items of work. Some processes, 
or operations as we agreed to call them, break into 
stages in a natural way; in others the partitioning, 
(for the sake of DP analysis) is performed artificially 
as, say, in aiming a missile which can be conditionally 
broken down into stages each of which takes some 
time to be accomplished. 

Assume further that the operation in question con¬ 
sists of m stages. Let the effectiveness of the operation 
be characterized by a performance measure, W, which 
will be succinctly referred to in this chapter as reward , 
or return. Assume that the total return of operation, 
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W, makes up of the returns at separate stages, w t , 

m 

W = (4.1-4) 

2=1 

i.e., has the property of additivity. 

The concerned operation is a controlled process which 
means that we can choose the parameters affecting its 
performance and final result. Since it is a sequential 
decision process, at each stage we may select a decision 
upon which the immediate result depends at this stage 
and the total return of the process. The set of all se¬ 
quential decisions constitutes a policy pursued on the 
process as a whole. Let x represent a policy, and x t 
an immediate decision at stage i, then 

x = [aq, x 2 , . . ., x m ] (4.1-2) 

It should be kept in mind that the decisions are not 
necessarily numerals, they may be vectors, functions, 
or other mathematical forms. 

The goal of DP is to establish a sequence of decisions, 
x, such that maximizes the total return W: 

m 

W = 2 w t =^~max (4.1-3) 

2=1 

The sequence of decisions x* which yields the 
maximum gain will be called the optimal policy (or the 
optimal control, if one wishes it in terms of control). 
The sequence consists of optimal decisions made each 
at its stage: 

x* = [x* v x* 2 , . . ., x£] (4.1-4) 

The maximal reward which is earned with this policy 
will be referred to as W *, that is, 

W* = max [fP (x)J 


(4,1-5) 
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which is read as: the value W* is maximal of all W (x) 
taken over all possible policies x. A constraint imposed 
on the policies, that is, the fact that they must belong 
to a certain set X, is occasionally mentioned in the 
formula, namely, 

W* = max [kE(x)] (4.1-5') 

xex 

Consider several examples of sequential decision 
processes to display what is meant in each of them 
under the term ‘policy’ and what is the return (measure 
of effectiveness) W. 

EXAMPLE 1. A financial activity of a group of 
plants is planned for an m-year planning horizon. 
The company that controls the group supposes to 
invest in its development some monetary fund (re¬ 
sources) which must be allocated among the plants. 
As the time elapses, the resources invested into a plant 
are known to be partly expended (depreciated) and 
partly remained available for further reinvestment. 
Each plant makes a profit depending upon the amount 
of invested capital. At the beginning of each fiscal 
year all the available resources are reallocated among 
the plants. What amount of capital must be assigned 
to each plant at the start of a year to realize a maxi¬ 
mum profit at the end of the planning horizon? 

The total return (total profit) sums up of immediate 
profits over each year: 

m 

W = 2 w t (4.1-6) 

i =1 

and, hence, exhibits the property of additivity. 

The decision made at the ith stage is to assign at 
the beginning of the ith year some amounts of the 
capital x (1 , x ih (the second subscript refers 
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to a particular plant) to the plants. Thus, in this 
problem a decision is a /c-component vector: 

Xi (^iii ^12* • • *i %ih ) 

Each decision enters into a sequence x which is a 
policy for the entire operation 

x = [x t , x 2 , . . ., x m \ (4.1-8) 

The problem is to find an assignment of the capital 
to the plants and over years such that will maximize 
the total profit W. 

While in this example the decisions are vectors, in 
the subsequent examples they manage with numbers. 

EXAMPLE 2. A rocket has m stages so that its 
orbiting may be broken down into m steps at the end 
of which a stage burns out and separates. All the 
propelling stages together (i.e. without the spacecraft) 
have some stipulated weight 

G = <?i + G 2 + . . . + G m 

where G t is the weight of the ith stage. 

At the end of the ith leg of the escape trajectory 
(when the ith stage burns out and separates) a what- 
remains-of-the-rocket boosts its velocity by A ; , the 
increment depends on the weight of the separated stage 
and the total weight of both the remaining stages and 
payload of the spacecraft. What weight must be allo¬ 
cated to each stage so that the terminal speed be maxi¬ 
mal? 

The measure of effectiveness to this problem will be 
the orbiting speed 
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where A t is the return (speed increment) at the ith 
stage. A solution to the problem will be a set of weights 
of all stages 

x = [Gj, G 2 , . . ., G m ] 

The optimal solution, obviously, will be the distri¬ 
bution of weights which maximizes the speed V. 

EXAMPLE 3. A car owner prepares to use a car for 
m years. At the turn of each next year he would face 
the following alternative situation: 

(i) to sell the car and buy a new model, 

(ii) to have it repaired and use it further, 

(iii) or to use it without any repair. 

The decision now is a choice of one of the three 
alternatives. They do not relate to figures, but the 
first may be assigned the value of 1, the second 2, 
and the third 3. What decisions must be made each 
year (i.e. how the assignments 1, 2, and 3 must be 
alternated) to minimize the total expenses for use, 
repair, and buying a new car? 

The measure of effectiveness this time is pay-off 
rather than gain, but it does not matter. It may be 
expressed as 

m 

= S w t (4.1-10) 

i= 1 

where w t is the expenses for the /111 year. The quantity 
W must obviously be minimized. 

A solution (policy) to this problem is a combination 
of the three numbers 1, 2, and 3, e.g., 

x - [3, 3, 2, 2, 2, 1, 3, . . .) 

which is decoded as: the first two years use the car 
without repair, next three years it must be submitted 
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lo repair, at the turn of the sixth year seli it and 
buy a new one, then again use it without repair, and 
so on. Any policy therefore is a collection of numerals 

x = l/j, . . ., /J (4.1-11) 

where each of them,/j, is either of three numbers 1, 2, 
or 3. The problem is to choose a set (4.1-11) for which 
the quantity (4.1-10) is kept at a minimum. 

EXAMPLE 4. A stretch of railway is being con¬ 
structed between two locations A and B (Fig. 4.1-1). 



Fig.4.1-1 

It is to cross a rugged terrain ample with forest areas, 
hills, swamps, and a river calling for an inevitable 
bridge. The route must be chosen such that the total 
construction costs are a minimum. 

This problem in contrast to the previous has no 
natural partition into steps. It can, however, be effec¬ 
ted by breaking down the straight line section AB 
into m segments and drawing through the points of 
partition the lines perpendicular to AB. A step will 




110 


Ch. 4 Dynamic Programming 


then he visualized as a transition from one such line 
to the next. If the lines are many, i.c. drawn sub¬ 
stantially close to one another, then the segment of 
route at each slep may be deemed approximately 
linear. A decision at the Z111 step materializes as an 
angle cp* which the railway makes at this point with 
line AB. A solution to the problem consists of a 
sequence of all decisions 

X = ftpx, cp 2 , . . ., cpj 

It is required to choose an optimal sequence x* 
such that minimizes the total costs of construction 

m 

W = 2 u>,=Mnin (4.1-12) 

i=l 

Now that we have considered several examples of 
sequential decision processes, we may pause to discuss 
how these problems can be solved. 

Any sequential decision problem may be approached 
in two ways. We may either seek for all decision 
variables at all steps at once, or build an optimal 
policy step by step, optimizing only one decision at 
each computational stage. Normally the second ap¬ 
proach appears simpler than the first, especially when 
the number of steps is great. 

This idea of sequential, step by step optimization 
underlies the method of dynamic programming. One 
stage is, as a rule, simpler to optimize than the entire 
process: it proves more efficient to optimize many times 
an easy to solve problem than once attack a complica¬ 
ted hard nut. 

At first sight the idea might seem fairly trivial. 
Nothing seems more simple than to decompose a 
problem into a set of small steps when it is hard to 
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optimize as a whole. Each such step will bo an indi¬ 
vidual, small, and easy to optimize operation. All 
we need is to take a decision such that the effectiveness 
of this step is optimal, isn’t it? 

Not at all! Way off target! The basic concept 
of dynamic programming has never suggested that 
each stage be optimized separately of the others. To 
the contrary, each decision must be selected with 
account of all its sequences in the future. What is 
the use of choosing a decision which maximizes the 
return at this stage if it deprives us of a possibility of 
a good reward at the subsequent stages? 

Consider, for example, a planning for a group of 
manufacturing plants of which ones manufacture some 
merchandise while the others produce the relevant 
manufacturing equipment. The objective of this oper¬ 
ation is to manufacture for m years of the planning 
horizon a maximum amount of the merchandise. 
Assume that we schedule a capital investment for the 
first year. Viewing under the narrow angle of the 
objective for this year, all the available capital must 
have been invested into manufacturing of the mer¬ 
chandise. Will, however, this decision be efficient 
from the viewpoint of success for the whole activity? 
Obviously not, because this decision is wasteful and 
shortsighted. With the future in view, we must have 
invested a part of the capital into machinery. This 
surely would diminish the produced volume of mer¬ 
chandise for the first year, but would create the pro¬ 
duction capacity for its increase in the years to come. 

Another example to the point. Suppose that in the 
problem on routing a railway construction from A 
to B we had been tempted with the easiest starting 
direction and started constructing on this path. What 
is the use of the easiness and economy on the first 
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leg il' the very next would run the way into a swamp? 
It all implies that 

planning a sequential decision process we need to 
choose a decision at each stage with due con¬ 
sideration for its consequences at the stages to 
come. 

A decision at the ith stage is taken so as to maximize 
the total return for all the subsequent stages plus this 
one , rather than with the point to maximize the return 
at this stage only. 

There are no rules without exception, however. 
Among all the stages there is one which can be adopted 
without consulting the future consequences. Which one 
is this? Obviously the last. This stage is the sole to 
have a decision invoking the largest reward for it 
alone. 

Therefore dynamic programming works normally 
backwards, reeling up from the terminal step to the 
beginning. The first decision, thus, is made at the last 
stage. How can it be made, however, if we do not know 
what was the result of the previous, the last but one 
stage; that is without a knowledge on the conditions 
under which we have to decide on the last step? 

This is the point where lies the essence of the pro¬ 
cedure. Working of the last stage, one needs to make 
various suppositions concerning what was the result 
of the last but one, (m — l)st step and then to devise 
a conditional optimal decision at the mth step (‘con¬ 
ditional’ refers to the fact that the decision is made 
under the condition that the previous step has resul¬ 
ted so and so). 

Suppose we have done it and know for each of pos¬ 
sible outcomes of the last but one stage its conditional 
optimal decision with the attendant optimal return 
at the mth stage. Then we can optimize the decision 
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of the previous (m — l)st stage. Again make all pos¬ 
sible suppositions on the outcomes of the previous, this 
time (m — 2)nd stage and on the se suppositions 
find a decision at the (m — l)st stage such that maxi¬ 
mizes the return at the two last stages (of which the roth 
has been already maximized). In this way we find for 
each outcome at the (m — 2)nd stage the respective 
conditional optimal decision at the (m — l)st stage 
and the attendant optimal return at the two last stages. 
If the same process is continued we can back up all 
the way to the beginning, finding optimal decisions 
as we go. 

Assume that all conditional optimal decisions togeth¬ 
er with the attendant returns at all the subsequent 
stages, i.e. from the given stage to the end, are known. 
It means that we know what to do, how to decide at 
this decision point, and what will be the resultant 
return at the end whatever the state of the process at 
the beginning of this stage. Now we are in a position to 
discard the attribute ‘conditional’ building a simply 
optimal policy, x*, and evaluating an optimal total 
return, W*, to substitute those derived earlier as 
‘conditional’. 

To prove, assume that we know the state S 0 of the 
controlled process at the first stage. Then we can make 
an optimal decision x* at this decision point. It will 
change the state of the process to a new one, say S* % . 
In this state the process arrives at the second stage. 
This, hence, gives us the knowledge on the conditional 
optimal decision x* which translates the process to 
a stage S* at the second stage, and so on. As to the 
optimal total return W* for the entire operation, we 
know it already, since it has been its maximality that 
we have in view making the decision at the first 
stage. 
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Consequently, in optimizing a policy by dynamic 
programming, a sequential decision process is as if 
“run” twice. First moving backwards to find condition¬ 
al optimal decisions and conditional optimal returns 
for the “tail” stages of the process. Then in normal 
order when all we need is to read the prepared recom¬ 
mendations and evaluate the unconditional optimal 
policy x* as a sequence of optimal decisions x*, 
r * 

x 2 ’ ■ • *i • 

The first run, that of conditional optimization, is 
much more involved and time consuming of the two. 
The second run requires almost no additional computa¬ 
tion. 

The author does not flatter herself with the hope that 
the reader who has never had a chance to meet dynamic 
programming before might get a true understanding 
of its basic idea from the above presentation of the 
method. A more comprehensive grasp could hopefully 
be obtained in considering examples of value. Some 
of them will be discussed in the subsequent section. 

4.2 Solving Dynamic Programming Examples 

This section is devoted to consideration (and even com¬ 
plete solution) of several simple (purposely simplified) 
dynamic programming problems. 

A cost-efficient path problem. Recall the railway con¬ 
structing problem of the previous section to solve it 
completely under strictly (and purposely) simplified 
conditions. Resuming, we need a path connecting two 
locations A and B of which the latter is situated, say, 
northeast of the former. To simplify the model, assume 
that the tracing is broken down into a series of steps on 
each of which we may move either east- or northwards. 
Therefore any path from A to B is a broken line whose 
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sections are parallel and perpendicular to either of 
coordinate axes (see Fig. 4.2-1) The costs to complete 
each of such sections are known. How to route a path 
from A to B to keep the total costs at a minimum? 



We may adopt either of two ways: to compare by an 
exhaustive search all the possible alternatives and 
choose that of minimal costs (it would be a very difficult 
procedure when the number of sections is large), or to 
break down the process of moving from A to B into 
individual steps (one step equivalent to one line seg¬ 
ment) and then optimize the path step by step. The 
second way turns out far more convenient. Here, as 
elsewhere in operations research there tells the superi¬ 
ority of direct search over naive exhaustive routine. 

We illustrate the procedure by working it. Divide 
the distance from A to B in the eastern direction into 7 
segments, say, and in the northern direction into 
r> segments (basically, the division may be arbitrarily 
frequent). Then any path AB will consist of m = 7 
|-5 = 12 segments pointed either east or north. 


8* 
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Figure 4.2-2 displays the network, showing the cost of 
construction (in some conventional units) on each 
segment. It is required thus to select a path AB for 
which the sum of segmental costs is minimal. 
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Fig. 4.2-2 


The constructed path may also he visualized as a 
movement of a controlled system subject to issued deci¬ 
sions from the original state A to the terminal state B. 
The state of the system before moving to a next leg will 
be described by two coordinates: the eastern ( x) and 
the northern ( y ), both of which assuming only integer 
values (0^ 7, 0^ 5). For each state of the 

system (a node in the network of Fig. 4.2-2), we are to 
derive a conditional optimal decision: either go north 
(decision N) or go east (decision E). The decision is 
made so that the cost of all remaining legs (including 
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the given step) is minimal. We will customarily refer to 
this cost as ‘conditional optimal return’ (although in 
this case it is a pay-off, not a gain) for a given state of 
the system before entering a next leg. 

First perform conditional optimization moving back¬ 
wards. Starting with the last, twelfth step, consider 
(he upper right corner of the network, cut out in 
Fig. 4.2-3. Where could we be after the 11th step? 
Obviously, either in B x or B 2 . If in jB x , then we have no 
choice and go east; it will cost 10 units. Put this cost 
into the node circle of B x and show the respective 
optimal decision by a short arrow pointing from the cir¬ 
cle eastward. For node B 2 , the decision is also enforced: 
go north, which entails a cost of 14 units encircled 
and arrowed at B 2 . The conditional optimization of the 
last step is effected and the attendant conditional opti¬ 
mal return is evaluated and encircled at the respective 
node. 



Now we come to optimize the prior, 11th step. After 
the 10th step we could turn out at one of the nodes C x , 
C 2 , or C 3 shown in Fig. 4.2-4. Let us evaluate for each 
of them their conditional optimal decisions along with 
attendant returns. At node C x the decision is obvious: 
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go east with the cost to the end equal to 21 units. We 
home this figure into the circle at C v For C 2 , the 
decision is no longer enforced: we may go both east 
and north. The first route entails a cost of 28 units 
summed up of 14 toward B 2 and another 14 to the end. 
If we go north, the route totals 13 + 10 = 23 units. 
This implies that the conditional optimal decision 
at node C 2 is bound northwa r d, which is marked by 
the arrow and the number 23 in the circle of node C 2 . 
As to C 3 , the decision is again enforced: go north (N) 
with a cost of 22 units. 

Working backwards in a similar manner, we can find 
for each node its conditional optimal decision (N or E), 
put the respective return to the circle with an arrow 
pointing in the direction of departure. The return each 
time is evaluated on a recursive scheme; the return at 
a given step sums up with that already optimized and 
written in the circle to which points the arrow. There¬ 
fore, at each stage we optimize only the step under 
concern while the following are already optimized. 
The procedure results in the network presented in 
Fig. 4.2-5. 

With the completed conditional optimization, we 
know where to go (an arrow) and what it will cost in 
total (figure in a circle) at whatever node we might turn 
out. The node circle at A contains the optimal return 
for the whole path AB: W* = 118 units. 

All what is left now is to evaluate the unconditional 
optimal solution, that is, the path leading from A to B 
in the cheapest way. To do this we need only to go in 
the direction shown by the arrows. The optimal path 
is indicated in Fig. 4.2-5 by dual circles. The respec¬ 
tive optimal policy is seen to be 


x* = [N, N, N, N, E, E, N, E, E, E, E, E] 
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that is, the first four steps must be to the north, next 
two to the east, then one step to the north, and the five 
remaining to the east. This completes the solution. 



Note that in a process of conditional optimization 
we may face a tie case when two or more decisions are 
optimal for a node, i.e. result in equal costs (expendi¬ 
ture of resources) from this node to the terminal node. 
In the above example, the node having coordinates 
(5, 1) enjoys two optimal decisions, N and E, resulting 
in a remaining cost of 62. We may choose any of them 
(we have preferred N but could equally have made 
lor E). These cases of tie decision making are recurrent 
phenomena in dynamic programming. We shall not 
pause on them in the future, simply picking up one of the 
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equivalent options. This deliberateness can, obviously, 
affect an optimal policy, not the optimal return. 
Dynamic programming problems in general (as well as 
linear programs) have a unique solution not always. 

Now return to our sample problem and try to solve 
it by a “naive” approach, choosing at each step, start¬ 
ing from the first one, the most cost-efficient (for 
this step) direction (if two, then any will go). This 
approach gives rise to the solution 

x = [N, N, E, E, E, E, N, E, E, E, N, N1 

The costs for this path are W = 10+ 12 + 8 
+ 10 + 11 + 13 + 15 + 8 + 10 + 9 + 8+14 = 
128, which is undoubtedly more than W* = 118. 
The difference in this case is not very strong, though 
it might be far more substantial. 

In the problem just solved the conditions were pur¬ 
posely submitted to a severe simplification. Certainly, 
it occurs to no one to trace a railway in steps, moving 
either strictly north or strictly east. This simplification 
was made solely to choose at each node out of two 
decisions only: N or E. We can, certainly, take more 
alternative directions and shorten the step; this will 
not make any principal difference, though, will make 
the computations more complicated and lengthy. 

The problems similar to the discussed example can 
very often be met in applications, for example, in 
choosing the quickest path between two points, or most 
economical trajectory (in terms of fuel consumption) 
for an ascending aircraft aiming at specified altitude 
and speed. 

Let us stop for a moment to discuss an interesting 
aspect. The observant reader might have noticed that 
in the above example problem, the origin and terminal 
points A and B do not principally differ from one 
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another. A conditional optimal solution might be 
deduced moving from the beginning to the end, rather 
than backwards, while the unconditional path on the 
back swings to the origin. It is indeed the case for any 
DP problem because the start and end may be inter¬ 
changed. The resulting procedure will be entirely 
equivalent in the computational aspect to that de¬ 
scribed above, but somewhat less convenient to describe 
in words its ideas because it is easier to argue referring 
to the conditions which have already been created prior 
to the beginning of this stage than to those which are 
yet to come. In essence though, both approaches are 
equivalent. 

A resource allocation problem. Dynamic program¬ 
ming proves very powerful technique in solving to 
advantage many economic applications (see, for exam¬ 
ple, [6,10]). Consider a sample problem of this variety. 

We (a board of directors) have at our disposal some 
capital funds (resource) which we are going to invest 
in, or allocate among, m ventures. Each of the ventures 
is known to produce when invested with a capital 
x a profit which is a function of the investment, f t ( x)■ 
All the functions fi(x)(i = i,2,...,m) are specified 
(they are certainly nondecreasing). How should we 
allocate the funds among the ventures to realize a 
maximum profit? 

This problem can be readily handled by dynamic 
programming. Although the problem does not contain 
in its statement any time periods, the financial opera¬ 
tion can be thought of as a sequence of transactions, 
assuming the first stage to be the investment in ven¬ 
ture 1, the second stage the investment in venture 2, 
and so on. 

We may again invoke the notion of a controlled sys¬ 
tem, S, which in this case may be visualized as the 
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funds or resources to be allocated. The state of the 
system S is characterized at each decision point by 
a single number S representing the available, not yet 
invested, capital. The decisions in this problem are 
the amount of resources x lt x 2 , . . ., x m allocated to 
the ventures. It is required to devise an optimal policy, 
that is such a sequence of decisions x which maximizes 
the total profit: 

m 

W ~ 2 /,• (£j) =>-max (4.2-1) 

i=l 

We shall approach the problem first generally to 
derive the needed formulas and then solve it with 
numbers. Find for each ith stage a conditional optimal 
return (from this stage through all the remaining stages 
to the end) if we have arrived at it having an available 
amount of resources S. Let W t ( S ) represent the con¬ 
ditional optimal return, and x t ( S ) the respective 
conditional optimal decision, i.e. the amount of re¬ 
sources invested in venture i. 

Begin optimizing from the last, mth stage. Assume 
that we have arrived at this stage with a remained 
amount of funds S. What should we do? Obviously 
invest the entire funds S in venture m. Therefore, the 
conditional optimal decision at the rath stage will be: 
invest in the last venture all the available resources S, 
i.e., 

(S) = S 

with the conditional optimal return 

(S) = fm (S) 

Letting S assume a series of densely spaced values, we 
may compute x m (S) and W m ( S) for each of them. The 
last stage may be deemed optimized. 
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Now go over to the previous, (m — l)st stage. Sup¬ 
pose that we have come to this decision point with an 
available amount of resources S. Let (5) refer 

to a conditional optimal return at the two remaining 
stages (m — l)st and /nth (which has been already 
optimized). If we assign to the (m — l)st venture 
a capital x, then the last venture receives the remaining 
S — x. The return at these two stages will be 

fm-l (*) + W m (S — x) 

and we must find an x such that maximizes the return 
W m -1 (S) = max [/ m _, (*) + W m (S — *)] (4.2-2) 

Notice that we maximize over all possible values of x 
which, however, cannot be larger than S because we are 
unable to invest more than S. This maximum is the 
thought conditional optimal return at the last two 
stages, and the value of x at which it is reached is the 
thought conditional optimal decision at the (m — l)st 
stage. 

Stepping further back we optimize (m — 2)nd, 
(.m — 3)rd and earlier stages. In general, the condition¬ 
al optimal return from all stages starting from a given 
i to the end will be found with the formula 

W t (S) = max [/, (x) + W i+i (S - *)] (4.2-3) 

x<S 

The respective conditional optimal decision x t (S) 
will be that value of x at which this maximum is at¬ 
tained. 

Proceeding in the same fashion, we shall arrive 
finally at venture 1. Here we have no need to vary S 
since we know exactly the total amount of available 
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resources prior to entering the first stage, R, therefore 
W* = (R) = max [/, (*) + W t (R - x)] (4.2-4) 

Hence, the maximal return (profit) from all ventures 
is found. Now we need only to “read the recommenda¬ 
tions”. The value of x at which (4.2-4) is a maximum 
relates to the optimal decision x* at the 1st stage. 
Having invested these funds in the 1st venture, we are 
to decide on the remaining R — x*. “Reading” the 
recommendation for this value of 5, we allocate to the 
second venture its optimal amount x* = x 2 (R — 
— x\) and so forth, to the end. 

Consider now a numerical example. An input amount 
of resources valued R — 10 units must be optimally 
allocated to m — 5 enterprises. For simplicity assume 
that only integer amounts are invested. The functions 
of return are summarized in Table 4.2-1. 


Table 4.2-1 


X 

h (*) 

/* (*) 

/ 3 (*) 

ft (*) 

h (*) 

1 

0.5 

0.1 

0.6 

0.3 

1.0 

2 

1.0 

0.5 

1.1 

0.6 

1.2 

3 

1.4 

1.2 

1.2 

1.3 

1.3 

4 

2.0 

1.8 

1.4 

1.4 

1.3 

5 

2.5 

2.5 

1.6 

1.5 

1.3 

6 

2.8 

2.9 

1.7 

1.5 

1.3 

7 

3.0 

3.5 

1.8 

1.5 

1.3 

8 

3.0 

3.5 

1.8 

1.5 

1.3 


A common feature that might strike in all the col¬ 
umns is that starting with some level of investment the 
profit ceases increasing (in real life the phenomenon 
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corresponds to the fact that each enterprise is able to 
accommodate only a limited amount of investment). 

Perform conditional optimization in the manner 
described above, starting with the last, 5th stage. Each 
time we come up to a next stage with an available 
resource S, we allocate at this stage an amount of 
resource, look up the return for this stage in Ta¬ 
ble 4.2-1, add it up to the already optimized return at 
all the subsequent stages to the end (considering that 
the remaining funds are lowered exactly by the allocat¬ 
ed amount) and find the investment for which the 
sum maximizes. This investment relates to the condi¬ 
tional optimal decision at this stage and the maximum 
proper is the conditional optimal return. 


Table 4.2-2 


S ■ 

i 

= 5 

i = 

= 4 

i = 

= 3 

i 

= 2 

i 

= 1 

*6 ( S ) W 6 ( S ) *4 ( S ) 

W 4 (S) x 3 (S) W 3 (S) x 2 (S) W 2 (S) Xl (S) 

i Wx(S) 

1 

□ 

1.0 

0 

1.0 

0 

1.0 

0 

1.0 



2 

2 

1.2 

i 

1.3 

l 

1.6 

0 

1.6 



3 

3 

1.3 

2 

1.6 

PI 

2.1 

0 

2.1 



4 

4 

1 .3 

3 

2.3 

2 

2.4 

0 

2.4 



5 

5 

1.3 

3 

2.5 

1 

2.9 

1) 

2.9 



6 

6 

1.3 

4 

2.0 

2 

3.4 

5 

3.5 



7 

7 

1.3 

5 

2.7 

2 

3.6 

5 

4.1 



8 

8 

1.3 

5 

2.8 

4 

3.7 

ill 

4.6 



9 

9 

1.3 

6 

2.8 

5 

3.9 

7 

5.1 

a 


10 

10 

1.3 

7 

2.8 

5 

4.1 

7 

5.6 

13 


Table 4.2-2 summarizes the results of conditional 
optimization over all stages. The table is seen to be 
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set up from the upper left corner to the right and then 
line by line downwards. Tire decision at the 5tli stage 
is enforced: all the available funds are invested; the 
decisions at all other stages have to be optimized. 
The sequential backwards optimization from stage 5 to 
stage 1 yields a complete list of sequential decisions and 
the total optimal return W* for the whole operation, 
amounting to 5.6 in this case. The last two columns 
have only one row because the initial state of the 
system is known exactly: S 0 = R = 10. The optimal 
decisions at all stages are confined in frames. The 
optimal sequence of investments reads thus: allocate 
two units of 10 to enterprise 1, five units to enter¬ 
prise 2, two units to enterprise 3, none to enterprise 4, 
and a unit to enterprise 5. With this allocation the 
total profit attains a maximum of 5.6. 

To provide the reader with an insight to how the 
table was set up, we perform a sample computation. 
Suppose, for example, we are to optimize a decision 
x 3 (7), that is, to decide on how many units we should 
allocate at stage 3, where we have arrived with S — 7 
units of resource available, and what maximum amount 
we can earn at all the remaining stages including the 
third. Assume all the stages following the third (i.e. 
4th and 5th) have already been optimized and, hence, 
the columns for these two stages in the table are idled. 
Let us find x 3 (7) and W 3 (7). To effect this, compose 
an additional table (see Table 4.2-3). Its first column 
lists all possible investments x at the third stage, which 
cannot be more than S = 7. The second column lists 
all what remains from 5 = 7 after this investment. 
The third column presents the return at stage 3 from 
the investment of amount x in enterprise 3 (filled with 
reference to the column for/ 3 (a;) in Table 4.2-1). The 
fourth column gives the optimal return at the remaining 
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(4th and 5th) stages on the condition that we have 
come up to stage 4 with what has been left of the 
resources (filled with reference to column i = 4 of 

Table 4.2-3 


a: 7 -- x /:i (.%•) W,[ (7 — x) f s (x) + IV.| (7 — x) 


7 

0 

1.8 

0 

1.8 

6 

1 

1.7 

1.0 

2.7 

5 

2 

1.6 

1.3 

2.9 

4 

3 

1.4 

1.6 

3.0 

3 

4 

1.2 

2.3 

3.5 

13 

5 

1.1 

2.5 | 

3.6 

1 

6 

0.6 

2.6 

3.2 

0 

7 

0 

2.7 

2.7 


Table 4.2-2). The fifth column lists the sum of two 
returns: that attendant to the decision made at the 
stage and the optimized return from the following 
stages with the investment x at the third stage. 

Of all the returns in the last column, we pick up the 
maximal, IT., (7) = 3.0, which relates to the decision 
x (7) = 2. 

A situation with a maximum (in an additional table 
of the type) attained at two or more x rather than one 
can be easily handled by choosing any of them because 
this choice does not affect the reward. As we have already 
mentioned, the solution of dynamic programming 
problems need not in general be unique. 

A truck loading problem. The approach of dynamic 
programming can successfully be used to tackle certain 
optimization problems discussed in Chapter 3 # in par¬ 
ticular some problems of integer programming. Notice 
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that the integrity requirement impeding the solution 
of LP problems, this time relaxes rather than compli¬ 
cates the procedure (as did the integrity of investments 
in the previous problem). 

Consider by way of example a problem on loading 
a truck (a similar loading problem was treated in the 
previous chapter) which formulates as follows. A truck 
can carry a total of Q tons. A certain collection of items 
(a sole thing of each) is available for shipment. Their 
weights and values are known Isay, tabulated as in 
Table 4.2-4). Determine which of the items must be 
shipped to maximize the value of the loading which 
is ^ Q. 

As can be readily observed, this problem in essence is 
very much the same as the previous (resource allocation 
among n enterprises) but is somewhat simpler than that. 
To demonstrate, the process of loading can be thought 
of as consisting of n stages; at each of them we decide 
whether or not to take a given item in the truck. The 
decision at stage i is unity if we take the ith item, and 
zero if not. It means that at each stage we have only 
two decision alternatives and this is a very comfort¬ 
able thought. 

The state of the system prior to each next stage will 
obviously be described by the loading capacity, S , 
which still remains after some items have been loaded 
to the truck. For each value of S we should find the 
total maximal value W t ( S ) of the items which still 
can be loaded on at a given value of S, setting each 
time Xi ( S) = 1 if we take the item, and x t ( S) = 0 
if not. Then we need to read these conditional recom¬ 
mendations, and the thing is done. 

Now solve this problem in numerical form for six 
items whose weights and values are specified in Ta¬ 
ble 4.2-4. 
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Table 4.2-4 


Item 

1 

2 

3 

4 

5 

6 

Weight 

4 

7 

11 

12 

16 

20 

Valuo 

7 

10 

15 

20 

27 

34 


The total loading capacity of the truck is Q — 35 
weight units. It is required to point those items which 
must be included in the loading to maximize its 
value. 1 

As before, S will be assigned only integer values. 
The conditional optimization of solution is indicated 
in Table 4.2-5, where each row gives for a respective 
stage (item) its conditional optimal decision x t (0 
or 1) and conditional optimal return Wi (the value of 
all the items remaining to be loaded at optimal deci¬ 
sions at all stages). We shall not dwell on how this table 
is prepared since the, procedure parallels that of the 
previous problem, the only difference being 0-1 num¬ 
bered decisions. 

The bold-faced values in Table 4.2-5 are: the opti¬ 
mal return W* = 57 and the optimal decisions provid¬ 
ing for it, viz., x* = 0, 3* = 1, x* — 0, x* — 1, 
x* = 1, x* =0. The solution thus is to load the truck 
with items 2, 4 and 5 whose weight totals 35 (the exact 
coincidence of weight and loading capacity is not 


1 The items in Table 4.2-4 are ranged in the order of increas¬ 
ing weights; their values also increase from left to right which 
is natural but not necessary. It is intuitively obvious that load¬ 
ing by high-weight and low-value items would be unsound. 


y-0432 
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Table 4.2-5 



i -• 

= 6 

■ 

mm 

Hi: 

i = 

= 4 

i = 

= 3 

i = 

= 2 

i - 

= l 

s 

x i 

W i 

x i 


x t 


x t 

w. 

x i 

M 'i 

x i 

w i 

0 

n 

0 

0 

0 

0 

0 

0 

0 

0 

0 



l 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



4 

0 

0 

0 

0 

0 

0 

(1 

0 

0 

0 



5 

0 

0 

0 

0 

0 

0 

0 

0 - 

0 

0 



6 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 



7 

0 

0 

0 

0 

0 

0 

0 

0 

1 

10 



8 

0 

0 

0 

0 

0 

0 

(1 

0 

1 

10 



9 

0 

0 

0 

0 

0 

0 

0 

0 

1 

10 



10 

0 

0 

0 

0 

0 

0 

II 

0 

1 

10 



11 

0 

0 

0 

0 

0 

0 

1 

15 

0 

15 



12 

0 

0 

0 

0 

H 

20 

0 

20 

0 

20 



13 

0 

0 

0 

0 

H 

20 

0 

20 

0 

20 



14 

0 

0 

0 

0 

H 

20 

0 

20 

0 

20 



15 

0 

0 

0 

0 

1 

20 

0 

20 

0 

20 



16 

0 

0 

1 

27 

0 

27 

0 

27 

0 

27 



17 

0 

0 

1 

27 

0 

27 

0 

27 

0 

27 



18 

0 

0 

1 

27 

0 

27 

0 

27 

0 

27 



19 

0 

0 

1 

27 

0 

27 

0 

27 

1 

30 



20 

1 

34 

0 

34 

0 

34 

0 

34 

0 

34 



21 

mm 

34 

0 

34 

0 

34 

0 

34 

0 

34 



22 

H 

34 

0 

34 

0 

34 

0 

34 

0 

34 



23 

H 

34 

0 

34 

0 

34 

1 

35 

H 

37 



24 

H 

34 

0 

34 

0 

34 

1 

35 

H 

37 



25 

H 

34 

0 

34 

0 

34 

1 

35 

1 

37 



26 

Hi 

34 

0 

34 

0 

34 

1 

35 

1 

37 



27 

i 

34 

0 

34 

0 

34 

mm 

42 

1 

44 



28 

l 

34 

0 

34 

1 

47 

H 

47 

0 

47 



29 

l 

34 

0 

34 

1 

47 

0 

47 

0 

47 



30 

H 

34 

0 

34 

1 

47 

0 

47 

0 

47 



31 

H 

34 

0 

34 

1 

47 

1 

49 

0 

49 



32 

H 

34 

0 

34 

H 

54 

0 

54 

0 

54 



33 

i 

34 

0 

34 

H 

54 

0 

54 

0 

54 



34 

i 

34 

0 

34 

H 

54 

0 

54 

0 

54 



35 

i 

34 

0 

34 

H 

54 

0 

54 

1 

57 

Kfl 

57 
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necessary; an optimal choice can afford some underload¬ 
ing). 

In retrospect it is possible to observe that a simpler 
solution to this elementary example might have been 
to check all possible combinations of items for fitting 
in an exhaustive manner to choose dually that one 
whose value will be maximal. With a larger number of 
items, however, this would be a laborious way because 
the number of possible combinations increases rapidly. 
Dynamic programming cannot be hampered with any 
rise in stages: it entails only a proportional increase 
in amount of computations. 

4.3 A General Form of DP Problem. 

The Principle of Optimality 

The trivial DP problems considered above provide 
insight to the basic idea of the method: step by step 
optimization performed “conditionally” in one direc¬ 
tion and “unconditionally” in the opposite. Dynamic 
programming is a very powerful technique to optimize 
a control; it cannot be impeded by either the integrity 
requirement, or nonlinearity of the objective function, 
or the type of imposed constraints. However, in con¬ 
trast to linear programming, DP cannot be reduced to 
a single computational algorithm; it can be relegated 
into a computer only after respective formulas are de¬ 
rived which often is not a simple task. 

This section will display a sort of a “review for a new¬ 
comer” discussing DP problem formulation, approach to 
solution, form of writing, etc. 

The first question to answer in problem formulation 
is: what input parameters (sometimes called state 
variables) can the state of a controlled process (or sys¬ 
tem) be characterized with prior to each stage? A suc- 


9* 
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cessful choice of a set of these parameters often decides 
whether or not the optimization problem can be solved 
to advantage. In the three sample problems we solved 
in the previous section, the state of the process was 
described by a very small number of parameters: by 
two coordinates in the first example, and by one numer¬ 
al in both the second and the third. These ultrasimple 
problems, however, are not frequent occasions in 
practice. More common are those where the state of 
the system is described by many state variables so 
that an exhaustion of all decision alternatives and 
evaluation of an optimal decision for each of them be¬ 
comes a cumbersome procedure. It becomes even more so 
when the number of possible decision alternatives is 
large. Richard Bellman, who developed the concepts of 
partial optimization underlying dynamic programming, 
claimed these situations as being “cursed” by multiple 
dimension scourging not only dynamic programming, 
but also all the other optimization techniques. A com¬ 
mon method of solving DP problems is on the comput¬ 
er rather than by hand; however, even modern large- 
capacity computers cannot match many such problems. 
It is very important therefore to formulate a DP 
problem correctly and economically without overload¬ 
ing the formulation with minor details, and simpli¬ 
fying as far as possible the description of controlled 
system (i.e. model) to shorten the number of decision 
alternatives. The reader might infer from what has 
been said that success in dynamic programming in 
many respects depends on the skill and expertise of the 
OR researcher. 

The second task after having been completed with 
system description and list of decisions is to break it 
down into stages. Some fortunate situations provide 
it in the very formulation of the problem (say, fiscal 
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years in economic problems), but more often the parti¬ 
tioning has to be invented, as we have done it in the 
railway path problem of Section 4.2. Should we have 
not confined ourselves within the primitive model of 
only two decisions (N and E), it would have been 
more convenient to break the path into steps in anoth¬ 
er way, for example, assuming a step is a passing 
from one line section parallel to the ordinate axis to 
a next. Another partitioning approach would be to 
consider instead of straight line sections a set of con¬ 
centric circles or other curves extending from the ori¬ 
gin at A . All these approaches to shortest path selec¬ 
tion would inavoidably restrict a choice of possible 
directions. Regarding the steps as transients from one 
constant abscissa line to another, it is easy to realize 
that the available manifold of decisions does not pro¬ 
vide for back up, i.e. from an eastern to a western 
line. For a majority of applications such restrictions 
are natural (for example, a rocket launched off the 
earth and supposed to move through an efficient escape 
trajectory could hardly be imagined to tilt its engines 
and perform a “nip” down to the earth). One may meet, 
however, an entirely different situation. For instance, 
a path through a rough terrain (as in mountains) spirals 
appearing at some sequentially further points geomet¬ 
rically closer to a point of departure. DP problem 
formulation, in particular choosing a system of coor¬ 
dinates and a breaking-into-stages approach, should 
consider all reasonable constraints imposed on the 
decisions involved. 

What policy should we adopt towards the number of 
stages ml At first glance it may seem the larger m the 
better. This, however, is not entirely so. With increasing 
m the amount of computations also increases, which 
is not always justified. The number of stages should 
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be selected with account of two conflicting circum¬ 
stances: (i) the stage must be sufficiently short to enable 
a simple optimization procedure for the decisions to 
be made, and (ii) the stage must not be too short to 
avoid unnecessary computations which may only com¬ 
plicate the search for an optimal solution, rather than 
materially change the optimum found for the objective 
function. In any application, it is the “acceptable” 
solution, which may insignificantly differ from the 
optimal in the return W *, that is of interest, not 
a strictly optimal solution. 

We are now ready to state a general principle which 
is a basic one to the solution of all DP problems 
(occasionally also termed the principle of optimality). 
Whatever the state of a process concerned prior to 
a given stage in a sequence, a decision must be 
made such that maximizes the return at this 
stage summed with the optimal return at all the 
remaining stages. 

T le readers in command of usual courses of calculus 
and probability might comprehend this principle 
completely only if it follows a discussion of certain 
examples. That is the reason why the principle is 
presented not at the beginning of the chapter—which 
would be natural for a mathematician—but here, when 
the necessary examples have already been considered. 

To summarize, we review the steps of DP problem 
formulation which might be of value to a novice in the 
field. 

1. Select the parameters (state variables) which 
characterize the state of the controlled system at each 
decision point. 

2. Break down the operation into stages. 

3. Evaluate the set of decisions x t for each stage 
along with the imposed constraints. 
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4. Determine the return which will result from 
a decision it at stage i, if prior to that the system was 
in a state S, i.e. to state the return function 

Wi = ft ( S , x t ) (4.3-1) 

5. Determine how the state S of the system will 
change in response to a decision xt made at stage i. 
The system will assume a new state 

S’ = (pi ( S , xt) (4.3-2) 

The functions governing transitions (4.3-2) must also 
be written 2 . 

6. Put down the main recursive equation of DP 
( functional equation) yielding the conditional optimal 
return Wt (S) (beginning with the ith stage to the end) 
in terms of the already known function W,- +1 (S) 

Wt (S) — max [ft (S, x t ) + W i+i (<p 4 (S, £;))] (4.3-3) 

Xj 

To this return there corresponds the conditional opti¬ 
mal decision xt (S) at stage i (it should be emphasized 
that the changed state S' = <pj ( S , xt) must be substi¬ 
tuted for S in the already known function Wt +1 ( S )). 

7. Perform conditional optimization of the last, with 
stage, specifying a set of states S one stage apart from 
the terminal state and computing for each of them the 
conditional optimal return by the formula 

W m (S) = max [f m ( S , xj] (4.3-4) 

x m 

to find the conditional optimal decision x m (S) which 
maximizes this return. 


2 Notice that the arguments of (4.3-1), (4.3-2) are generally 
collections of numbers, i.e. vectors, rather than single numbers. 
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8. Perform conditional optimization for the (m — 
l)st, (m — 2)nd and earlier stages with (4.3-3), set¬ 
ting there i = (m — 1), (m — 2), and so on, and show 
for each of the stages the conditional optimal decision 
xi ( S ) maximizing the expression. 

Note that if the state of the system in the initial 
moment is known (which is the case as a rule), then 
there is no need to vary the state of the system at the 
first stage: the optimal return for a given initial state S 0 
can be found directly. The obtained result is the total 
optimal return for the entire operation 

W* = W t (S 0 ) 

9. Perform unconditional optimization of the solu¬ 
tion thus obtained by “reading” the corresponding 
recommendations for each stage. With the optimal 
decision x* = x 1 (S 0 ) made at the first stage change 
the state of the system by (4.3-2) to find the optimal 
decision at the second stage x*; repeat the cycle at all 
the subsequent stages to the end. 

* * * 

Pause to make a few additional general comments. 
So far we have considered only additive DP problems, 
i.e. those where the total return results as the sum of 
returns from sequential stages. Dynamic programming 
can, however, be also implemented on the problems 
with a multiplicative criterion, i.e. expressed as 
a product 

m 

W= [| w t (4.3-5) 

<=i 

(if only the rewards wt are all positive). The solution 
of these problems is identical to that of additive crite* 
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rion with the sole difference that (4.3-3) has a prod¬ 
uct sign X in place of the plus sign, viz., 

W, ( S) = max [/, (S, x t ) X W j+1 (<P* (S, *,))] (4.3-6) 

x i 

To conclude the chapter, a few words are in order on 
the DP problems dealing with infinite sequential 
decision processes. In some applications, the operation 
has to be scheduled for an indefinitely long-time period 
and an optimal solution to the problem may be wished 
irrespective of the number of the stage at which the 
operation terminates. A convenient model for such 
problems is infinite sequential decision process in which 
all the stages are equal in that none of them is termi¬ 
nal. This process obviously calls for the return func¬ 
tion ft and the state transition function <p* be inde¬ 
pendent of the number of stage. We refer the interested 
reader to classical work [10]. Generally, a more pro¬ 
nounced insight into dynamic programming can be 
gleaned from the texts [6, 10, 11, 7], 




Chapter 5 

MARKOV PROCESSES 


5.1 The Concept of the Markov Process 

So far we have considered mainly deterministic OR 
problems along with respective optimization methods. 
Henceforth to the very end of the book our concern 
will he OR problems under uncertainty. This chapter 
will be focused on a comparatively favorable case, that 
of stochastic uncertainty (see Section 2.2). The un¬ 
knowns in such a problem are random variables (or 
functions) whose probability characteristics either are 
known or may be inferred from experiment. Our present 
consideration will be mainly devoted to what we agreed 
to call primal problems of OR, i.e. to model develop¬ 
ment for certain random phenomena, touching only 
shortly on the inverse problems, i.e. solution-optimi¬ 
zation techniques, because as a rule they are sophisticat¬ 
ed. For stochastic problems, even model development 
is a hard task, let alone optimization. The majority 
of the problems cannot enjoy a simple mathematical 
model such that would enable us to hnd explicitly 
(analytically) the wanted quantities (effectiveness 
measures) as functions of the operation’s constraints 
and decision variables. In some special situations this 
model construction can, however, be succeeded. This 
is the case when the operation in question is (exactly 
or approximately) the Markov process or chain. 
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A definition of the Markov process is deferred for 
a while, to give way for a discussion on the stochastic 
process in general. 

Let a certain physical system change its state in 
time (go from one state to another) in a random, 
unknown beforehand manner. Then we will say that 
a random (stochastic) process runs in the system. 

The physical system may be materialized in a varie¬ 
ty of ways, say, as a device, a group of such devices, an 
enterprise, an industry, a living organism, a population 
of species, and so forth. Randomness and uncertain¬ 
ty, though in different extent, are intrinsic to majori¬ 
ty of actual systems. 

Consider as an example of such system a spacecraft 
being put into an orbit. The orbiting will unavoidably 
suffer from random errors, that is deviations from the 
specified trajectory, necessitating a correction to be 
performed (if it were not for these random errors, no 
correction would be needed). Hence a putting into an 
orbit is a stochastic process. 

Now dive closer to the earth, in the dense atmosphere. 
A physical system this time is an aircraft performing 
a flight at a specified altitude and to a certain route. 
Is this process a stochastic one? Certainly, yes, since 
the flight would experience the influence of atmospher¬ 
ic eddies and other fluctuations which may result in 
random disturbances and fluctuations in its performance 
and accuracy of the flight schedule (anyone who 
suffered from a rough air or a lengthy waiting at an 
airport due to delayed airtraffic would evidence to 
the point). 

One more example, this time of a system realized as 
a facility involving several devices which used to fail 
from time to time, and when failed being replaced or 
recovered. A process running in this system is undoubt- 
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edly random. We may find a lot of random processes 
in the world around us. Moreover, an example of 
a nonrandom process is more hard to invent than that 
of a random process. Even if we turn to our watches— 
a classical example of precise operation coined the 
attribute ‘clockwork’—the process is liable to random 
variations, whether lagging, running, or occasional 
stop. 

Does it mean that all the natural processes are stochas¬ 
tic? Strictly speaking, it is so since random fluctuations 
are intrinsic to any process. Yet as long as these 
disturbances are insignificant and influence the para¬ 
meters concerned only slightly, we may neglect them 
and treat the process as a deterministic, i.e. nonran¬ 
dom, one. We have to account for random factors 
when they directly concern us. For example, in sched¬ 
uling flights an airline company may discard random 
fluctuations of an airplane about its mass center, 
whereas a designer of an autopilot may not. A majority 
of processes with which we deal in physics or engineer¬ 
ing are essentially random but only some of them we 
consider as random—when we are in pressing need 
of that. 

Having substantially elucidated the notion of ran¬ 
dom process, now we come over to define the Markov 
process. 

A random process is referred to as Markov if for 
any moment of time t 0 its probability character¬ 
istics in the future depend only on its state at 
time t„ and are independent of when and how this 
state was acquired. 

This important definition is worth discussing in 
a little more detail. Let at a present moment of time t 0 
(refer to Fig. 5.1-1) the system in question assume 
a certain state S a . We view the process from the side 
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and know at t 0 the state of the system S 0 and the 
history of the process, i.e. all events happened at 



t <zt 0 . We naturally wish to know what will be in 
the future, at t > t 0 . Can we forecast it? An explicit 
forecasting is impossible because our process is random, 
hence unpredictable. Some probability characteristics 
of the process we, however, may evaluate. For instance, 
the probability that in a certain time the system 
will turn out in a state S x or will remain in S 0 , or some 
other event. 

The key point is that a Markov process (or chain ) 
substantially facilitates this probability forecast as 
compared to the non-Markovian. Basically it is so be¬ 
cause if a process at hand is Markov, then we may do the 
forecasting taking into consideration only the present 
state of the system, S„, and putting aside the process 
history, i.e. its behavior at f< £„. Or, putting it 
another way, the future in the Markov process depends 
on the past only in terms of the present. 1 


1 The Markov processes where the probability of the next 
outcome depends only on the outcome immediately before are 
referred to as the first-order (or ‘lag-1’) Markov chains. If the 
outcome depends on other than the immediately prior result, it 
is a higher-order chain. For example, a second-order chain 
describes a process in which an outcome depends upon the 
two previous outcomes, etc.— Translator's note. 
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The Markov process may be exemplified by a Gei- 
ger-Mfiller counter which performs each next count 
when a nuclear particle enters the chamber. The state 
of the system’s display is defined by the indication, 
i.e. the number of particles which struck the tube 
prior to that moment. Let the counter indicate S 0 at t 0 . 
The probability that the count at t > t 0 will be 
particles (or not less than Sj) does depend on S 0 , but 
is independent of the particular instants at which 
previous particles entered the tube. 

Some recurrent applications may be modeled, if not 
exactly then to certain approximation, by the Markov 
chain. To illustrate, consider an air combat. The state 
of the battle is characterized by the number of aircraft 
remained in the battle (not shot down) x for opponent 
I and y for opponent II by a certain time. Assume 
that at t 0 we have the data on both parties, x 0 and y 0 . 
We would like to know the probability that battling 
airplanes I will outbalance by a moment t 0 + t v 
What does this probability depend on? First of all, on 
the state which the system occupies at f 0 ; it almost 
does not depend on when and in what sequence air¬ 
planes were shot down prior to t 0 . 

Now that the Markov process became a more or less 
evident model we are going to expose it from an unex¬ 
pected side: virtually any process may be modeled as 
Markovian if all the parameters of the “past” on which 
the “future” depends be included in the “present”. 
Imagine, for example, an item in use in an industrial 
context, say, which is subject to failure and prompt 
restoration from time to time; at a time f 0 it is still 
healthy and we wish to know the probability that it 
will be sound for another period t v If the present state 
of the item is thought of simply as ‘the item is sound’ 
then the process is obviously non-Markovian because 
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the probability that it will not fail for another peri¬ 
od t x depends generally on how long it has been already 
operating and when it was submitted to the last repair. 
If both these parameters (the total operation time and 
the time elapsed from the last failure) were included 
in the present state of the item, the process might have 
been deemed Markov. However, this boosting of the 
present at the expense of the past need not always be 
useful since (if the number of historic parameters is 
great) not infrequently it increases the size of the 
problem making it not amenable to available computa¬ 
tional facilities. Therefore in what follows, speaking 
of the Markov chain we shall think it of as a simple, 
non-subtle process having a small number of parame¬ 
ters defining the “present”. 

Markov processes in a “pure” form are non-common 
in applicational areas, though, non-seldom are those 
for which the effect of “history” may be neglected, and 
hence Markovian modeling be used to advantage. In 
not a long while we will demonstrate how it can be 
done. 

Stochastic processes may be loosely classed according 
to the type, discrete or continuous, their time variable 
and outcomes (states) belong to. Of all possible combi¬ 
nations of discrete (continuous) state/discrete (contin¬ 
uous) time processes we will be interested in the 
discrete-state!continuous-time process possessing the Mar¬ 
kov property as having great significance in OR appli¬ 
cations. We can imagine this process as that in which 
all the possible states can be indexed in advance and 
the moments of transitions from one state to the 
next are not fixed beforehand (as in the case of process¬ 
es developed in discrete time), rather they are random 
so that a transition may occur virtually at any time 
and proceeds jumpwise, in no time. 
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This process may be exemplified by a facility con¬ 
sisting of two units each of which can randomly, i.e. at 
an arbitrary moment of time, fail, whereupon it is 
immediately submitted to repair (renewal) which also 
lasts a random period of time, unknown in advance. 
The possible states of the system can be indexed as 
follows: 

S 0 — both units are healthy operating, 

— first unit is being repaired, second healthy, 

5 2 — second unit is being repaired, first healthy, 

5 3 = both units are being repaired. 

The transients from one state to another occur virtu¬ 
ally in no time at random instants of failure or termi¬ 
nation of a renewal period. 

A convenient approach to analysis of discrete-state 
stochastic processes is by invoking the geometrical 
analogy in the form suggested by graph theory. The 
states in a graph are depicted as some geometric 
figures (rectangles, circles or points) and called vertices, 
and the possible transients are shown by lines, called 
edges, or by arrows indicating the direction of a transi¬ 
tion, and then called arcs. We will display the states 
as rectangles placed in the vertices and containing 
symbols of states inside. 

A graph prepared for the above example is shown in 
Fig. 5.1-2. The arc leading from S 0 to implies the 
transition which occurs when the first unit fails; the arc 
pointing backwards, from to S 0 , signifies the 
transition at the moment of terminating the renewal 
of this unit. The other arcs can be assigned by similar 
argument. 

The astute reader might ask why an arc which would 
join S 0 and S 3 directly is absent. Can it not be that 
both units fail simultaneously, say, because of short cir- 
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cuit? A legitimate question. An answer is that we 
suppose the units fail independently of one another 



Fig. 5.1-2 

and the probability that they fail simultaneously is 
believed to be negligible. (This statement will receive 
a more explicit substantiation later.) 

If a process developing in the system with discrete 
states and continuous time is Markovian, then it can 
be described by a relatively simple model. However, 
prior to develop this model, we would find it useful 
to make familiar with the notion of ‘arrivals’ which 
is of significance for this and next chapters. 

5.2 Arrivals Defined 

A sequence of uniform events, called customers , which 
occur at random times, say, arrive at random at a ser¬ 
vicing facility, is referred to as arrivals, or an input 
( process ) to this facility. Examples may be arrivals of 
calls at a telephone exchange, arrivals of failures occur¬ 
ring within a computer, input of trains to a railway 
station, arrivals of nuclear particles at a Geiger- 
Miiller tube, etc. 

Arrivals may be visualized graphically as a series of 
points on the time axis Ot, shown in Fig. 5.2-1; one 
should not forget only that the position of each point is 
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random and Fig. 5.2-1 depicts a single realization of 
a process. 

Speaking of arrivals as a series of events that occur 
at random, one should keep in mind that the events, 

■c 

_ _I_ _ ______ «_, , !777%r~>. , . . . 

0 t 


Fig. 5.2-1 

realized say as a flow of customers, have no inherent 
probabilities. The probabilities are now in a possession 
of other, derivative events such as: “at time interval x 
exactly two arrivals occur” (see Fig. 5.2-1), or “at least 
one customer arrives in the interval At”, or “the inter¬ 
arrival interval will be equal or more than t". 

An important characteristic of arrivals is intensity 
(or arrival rate), X, stated as the mean rate of arrival 
per unit of time. The intensity may be constant or vari¬ 
able, say dependent on time t. For example, a stream 
of cars through a road crossing has a higher intensity 
at day time than at night, and is more intense in 
rush hours than in other hours of a day time. 

An input is called regular or that of constant arrivals 
(or intervals), if customers arrive in regular, equally 
spaced intervals. More common in applications are, 
however, arrival patterns whose inter-arrival intervals 
vary at random. 

An arrival pattern is termed stationary if its probabil¬ 
ity characteristics are independent of time. The 
intensity of this pattern need be constant. It does not 
mean at all that the number of customers arriving per 
unit time must be constant. The arrivals (if only they 
are not from a regular source) necessarily occur denser 
or rarer as, for example, is shown in Fig. 5.2.-1. 
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Important is that for a stationary input these density 
variations do not occur on a regular basis, and although 
one unit interval may enjoy more customers than 
another, the average number of customers per unit time 
is constant and hence independent of time. 

A typical mistake which a novice is liable to commit 
is to take random clusters and rarefactions of arrivals 
for variations in arrival rate. We ought to caution 
the reader on this point. 

As a rule, the deviations of a stationary behavior 
can be referred to some physical causes. It is natural, 
for example, that the arrival rate of calls arriving at 
a telephone exchange at night is lower than at day 
time (people used to sleep at night). More dense flow of 
customers arriving at a shop after working hours can 
also be explained. If an arrival pattern has a striking 
tendency to condense and rarefy (especially periodi¬ 
cally), then a physical cause might be suspected and 
its evaluation should be attempted. 

Arrivals of a stationary pattern (at least on a limited 
time interval) commonly arise in applications. To 
illustrate, arrivals of calls at a telephone exchange 
between 1 and 2 p.m. are virtually stationary; similar 
arrivals over the entire day would already be non¬ 
stationary. 2 

We have what is called an independent input process 
if for any two non-overlapping time intervals t x and t 2 
(see Fig. 5.2-2) the number of arrivals that occur in one 
of them is independent of that in the other. It implies 
essentially that the arrivals occur independently of 

2 Observe that the situation is the same with most processes 
which we used to refer to as stationary in physics and engineer¬ 
ing; actually they are stationary only over a restricted time 
interval and its extension to infinity is but a convenient tech¬ 
nique. 
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each other, due to its own cause each. For example, 
a stream of passengers entering a railway station is 
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virtually independent. Whereas a flow of customers 
leaving a counter in a shop cannot be deemed indepen¬ 
dent (at least because an interval of departure between 
any two of the customers cannot be shorter than a mini¬ 
mal time required to serve any one of them). It is 
similar with trains arriving at a station: there always 
exists a minimal time interval between each pair of 
them, established on safety grounds. Incidentally, if 
the minimal interval between arrivals is much less than 
the mean inter-arrival interval, t = 1/A., the effect 
of incomplete independence may be neglected. 

An input process is referred to as single arrivals if 
the customers arrive one by one, not in batches. To 
illustrate, a stream of customers arriving at a barber’s 
shop or queueing at a doctor’s office is normally of 
a single variety, which cannot be said of wedding 
parties arriving at a registry office in a large city. 
Arrivals of trains at a station are single, whereas 
arrivals of the cars are not. For single arrivals input, 
the probability of two or more customers to arrive 
within a small time interval At may be neglected. 

We will call an input Poisson if it possesses the 
three following properties at once: it is stationary, sin¬ 
gle, and general independent (i.e. is completely random). 
It is the simplest arrival pattern mathematically and 
the most commonly useful one in applications. By the 
way, the simplest arrival physically—the regular or 
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constant-intervals input—is not so simple to deal with 
mathematically since it lacks general independence: 
its arrival instants are related by a strong functional 
dependence. Such an input process cannot be created 



Fig. 5.2-3 


without special efforts to sustain its regular intervals 
of arrival. 

The role the Poisson arrival process plays among 
other arrival patterns is somewhat akin to that the Gaus¬ 
sian (normal) distribution does for the family of distrib¬ 
ution models. A superposition of a sufficiently large 
number of mutually independent, stationary, and 
single arrivals (of comparable intensity) yields an 
arrival pattern close to the Poisson scheme. 

It can be readily demonstrated (consult any text on 
probability, say [12]) that a Poisson input of arrival 
rate X has inter-arrival interval distributed exponential¬ 
ly with the density 

f (t) = X exp (—Xt) (t > 0) (5.2-1) 

shown in Fig. 5.2-3; here X is the parameter of the 
exponential distribution. For exponentially distribut¬ 
ed random intervals, the mathematical expectation, 
m, is the quantity inversely proportional to the para- 
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meter, and the standard deviation is equal to the 
mean: 

m = a = i/l (5.2-2) 

As a ‘measure of randomness’ for a nonnegative ran¬ 
dom variable, probability theory often exploits the 
coefficient of variation: 

v = a/m (5.2-3) 

From (5.2-2) and (5.2-3) it follows that for an exponen¬ 
tially distributed variable, v = 1, i.e. for the arrivals 
distributed in the Poisson process the coefficient of 
variation for intervals between arrivals is unity. 

Obviously, for a constant intervals input process 
whose inter-arrival interval is not random at all (a = 0), 
the coefficient of variation is zero. Most processes met 
in applicational areas have the coefficient of variation 
ranging between zero and unity, hence the coefficient 
may be employed as a measure of regularity, i.e. as 
a measure of deviation from the Poisson process. 
The closer it is to zero, the more regular is the input 
process. The Poisson process, being completely random, 
is the least regular of all applicational inputs. 3 

A convenient approach to arrival process computa¬ 
tions is in using what we shall call an ‘elementary pro¬ 
bability’. We will define it with reference to a Pois¬ 
son input of arrival rate X and a small time interval Af. 
An elementary probability is such that predicts at least 
one arrival at this interval. It can be readily shown 
(we will not go into that at length) that the elementary 
probability is 

p At = XAt + o (At) (5.2-4) 


3 One might invent a process for which v > 1, but nature 
almost never stages a state like that. 
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where the symbol o (At) denotes a quantity that becomes 
negligible compared with At, as At -vO. In other words, 
for a Poisson input, an elementary probability is 
equal to the arrival rate multiplied by the length of 
an elementary interval. Note that owing to indepen¬ 
dence, an elementary probability does not depend on 
whatever arrivals and when occurred earlier. 

Observe also that accurate to higher-order infinites¬ 
imals, the probability of at least one arrival in At is 
equal to the probability of exactly one arrival in this 
interval. It follows from the fact that the Poisson 
arrival process is single. 4 

We say that an input process is general independent, 
or recurrent, if it is stationary, single, and the inter¬ 
vals between arrivals, shown as T x , T 2 , T 3 , ... in 
Fig. 5.2-4, are independent and distributed with a gen¬ 
eral distribution function, say of a density as that in 
Fig. 5.2-5. Consider an example of an input process hav¬ 
ing a general independent form. A device, say a tube, 
functions continuously until it fails. A broken device 
is promptly replaced by a new one. If each subsequent 
device fails independently of the others, then arrivals 
of failures will be of a general independent form. 

Another example of this input process. A retailer in 
a shop continuously serves customers (as is the case in 
maximum load hours). Serving a customer takes a ran¬ 
dom period of time. The intervals between served cus¬ 
tomers will be recurrent if service times of the custo¬ 
mers are independent, that is, if a friendly chat be¬ 
tween the retailer and a customer cannot influence ser¬ 
vice times of others. 


4 Recall the discussion on absent arcs from S 0 to S 3 and 
back in the graph of Fig. 5.1-2. It may be explained by the 
assumption of both arrivals of failures and arrivals of renewals 
being single. 
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Obviously, the Poisson input is a particular case of 
the general independent process when the intervals 
between arrivals are distributed exponentially as in 



Fig. 5.2-4 

(5.2-1). Another degenerate case of the general indepen¬ 
dent input is the constant arrival process, whose con¬ 
stant intervals are not random at all. 

A whole family of recurrent arrival processes ordered 
to various degrees can be obtained fiy “screening” the 



Poisson input process. Let, for example, a stream of 
customers to an office be Poisson. Arriving customers 
are assigned at the reception board to clerks (servers) 
in strict rotation, i.e. the first customer to the first 
desk, the second to the second desk, and so on. If the 
office has n servers, then each of them would have to 
handle n-Erlangian arrivals of customers. The input 
process like that realizes from a Poisson process if we 
keep each nth arrival and screen off all the rest. The 
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Poisson input is but a first-order Erlangian arrival 
process. It may be demonstrated that with this screen¬ 
ing of a Poisson arrival process the respective coeffi¬ 
cient of variation for inter-arrival intervals reduces; 
and when the order n increases, the Erlangian input 
approximates to a regular process. The coefficient of 
variation for inter-arrival intervals of a ra-Erlangian 
input is z/ n > = 1/1 f n. The special Erlangian inputs 
form a family of arrivals variously ordered: from com¬ 
plete disorder (Poisson) to complete order (regular 
arrival process). 


5.3 The Kolmogorov Balance of State Equations 

In dealing with discrete-state/continuous-time Mar¬ 
kov chains, it would be convenient to think of the sys¬ 
tem’s transitions from one state to another as occurring 
under the influence of some arrivals (incoming calls, 
arrivals of failures, arrivals of renewals, etc.). If all 
the input processes translating the system from state 
to state are Poisson, then the process developing in 
the system is Markovian 6 . This is only natural, since 
the Poisson process is ‘memoryless’, i.e. its “future” 
is independent of the “past”. 

If a considered system is in a state Si from which 
it can directly pass into another state Sj (the transition 
is shown by the arc joining S t and Sj ), we will visualize 
the proeess as the system moved from Si to Sj by a Pois¬ 
son arrival process. That is, as soon as the first event 
of these arrivals occurs, the system “jumps” from St 
to Sj. 


1 A Poisson character of arrivals is a sufficient but not 
necessary condition for a process to be Markovian, 
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To be more illustrative, the respective graph ac¬ 
quires the intensities written each beside the arc of the 
respective transition. Let Xtj refer to the arrival rate 


“* 4 / 



Fig. 5.3-1 Fig. 5.3-2 


moving the system from Si to Sj. Then a resulting 
graph with all arrival rates indicated is shown in 
Fig. 5.3-1. (We shall call a graph with all transitions 
indicated a directed graph, or a digraph, for short.) 

Let us construct a digraph for the example of two ran¬ 
domly failing units in a facility, introduced in Sec¬ 
tion 5.1. As will be recalled, the states of the facility 
are as follows: 

5 0 = both units are sound, 

5 1 — first unit is being repaired, second is sound, 

5 2 = second unit is being repaired, first is sound, 

5 3 = both units are being repaired. 

We shall compute the arrival rates translating the 
system from state to state under the assumption that 
the mean time of repair does not depend on whether 
only one or both units are being repaired. It will be 
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the case when each unit is under the care of an individ¬ 
ual operator. Let the system be in S,,. Which arrival 
translates it in S x ?. It obviously is the arrival of failure 
occurring at the first unit. Its arrival rate is the 



Fig. 5.3-3 


inverse of the mean failureless time for this unit. 
Which process will back up the facility from S x to S 0 ? 
Obviously the arrivals of service completion for the 
first unit. The service arrival rate is the inverse of 
the mean time of repair for the first unit. The rates of 
other arrivals moving the facility through all the other 
arcs of Fig. 5.3-2 are computed in a similar manner. 

With reference to the constructed graph we may 
readily develop a mathematical model of this process. 

To demonstrate, consider a system having n possible 
states i S u S 2 , . . ., S n . The probability of state t, 
Pi ( t ), will be referred to occupying at time t state Si. 
An obvious fact is that for any time instant the sum 
of all state probabilities is unity, i.e. 

2 M*) = l (5.3-1) 

i=l 

With all arrival rates at hand one can obtain the 
probabilities of all states ( t) as functions of time. 
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For this purpose the Kolmogorov equations need be 
derived and solved. These are special differential equa¬ 
tions in state probabilities. 

A way to deriving these equations will be demonstrat¬ 
ed by means of an example. Let the considered system 
have four states » S lt S 2 , S 3 , and £ 4 ; the attendant di¬ 
graph is shown in Fig. 5.3-3. Consider one of the state 
probabilities, say p 1 ( t ). This is the probability that at 
time f the system is in state Sy. Giving f a small in¬ 
crement At find the probability p t ( t -f- Af) that at 
the time t + Af the system will be in Sy. It may, obvi¬ 
ously, happen in two ways: either ( 1 ) at the instant t 
the system was in Sy already and remained in this state 
for additional Af, or (2) at t the system was in S 2 and 
moved to S x during At. 

Find the probability of version 1. The probability 
that the system at the time t is in S u that is p x (f), 
must be multiplied by the probability that during At 
the system will not move to either S 2 or S 9 . The total 
input of arrivals shifting the system from Sy will also 
be Poisson with arrival rate X 12 + ^, 13 since a superpo¬ 
sition of two Poisson processes yields again a Poisson 
process because it retains the properties of stationarity, 
singleness and independence. Therefore the probabili¬ 
ty that for At the system will move from Sy is (X ia 
-f X 13 ) Af; the probability that it will not move is 
1 — (^12 + ^ 13 ) Af. Whence the probability of version 1 
is p r (f) [1 — (A , 12 + X ls )Af], 

The probability of version 2 is that at the time f the 
system will be in S 2 and for At will return back to S u 
i.e. p 2 (f) X 21 Af. 

Summing up the probabilities of both versions (ac¬ 
cording to the rules of probability addition) yields 


{>i (f + Af) = pi (f) [1 —(X 12 -f- Xy 9 )At] + p 2 {t)'K 2 iAt 
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Upon removing the brackets, transposing Pl ( t ) into the 
left-hand side, and dividing both sides by A?, we obtain 

= i 2if2 _ ^ 12 + x j3 ) Pl (t) 

Passing to the limit by letting At approach zero yields 
on the left-hand side the time derivative of Pl ( t). 
The resulting differential equation for Pl ( t ) is 

dpi ( t)ldt = k 21 p 2 (?) — (k 12 + l 13 ) Pl (?) 

or in a short form, without the argument ? of Pl and p 2 
which is no longer needed: 

dpjdt = ^ 2 iP 2 (^12 “h ^i 3 )Pi (5.3-2) 

Treating the other states with similar argument, we 
may derive three more differential equations which 
when added up to (5.3-2) give the system of differential 
equations in state probabilities: 

d Pl ldt — ^2i P 2 — (^12 “l - ^ 13)^1 

dpjdt = X 12 Pi ^32^3 (^24~t~ ^2l)p2 ^ 2 2^ 

dpjdt — ^3iPi -f* ^43^4 — ^32P3 

d Pi /dt — k 2 4 P 2 — 

We have obtained the set of linear differential equa¬ 
tions in four unknowns p x , p 2 , p 3 , and Pi . Note that any 
one of them can be eliminated by invoking the nor¬ 
malizing condition Pl + p 2 + P 3 + P 4 = 1 , i.e. by 
expressing one of the probabilities p t in terms of the 
others and substituting it for the respective probabili¬ 
ty in (5.3-3). 

We can now summarize the general procedure of 
deriving these equations. The left-hand side of each of 
the equations must contain the derivative of the pro- 
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bability of state i; the right-hand side must contain 
the multiple sum of the probabilities of the states lead¬ 
ing to the given slate ( i ) times the arrival rates of 
respective transients minus the total intensity of all 
arrivals shifting the system from the considered state 
times the probability of this state. 

Using this procedure, state the Kolmogorov equa¬ 
tions for the system whose digraph is depicted in 
Fig. 5.3-2: 

dpjdt = PjP! + p 2 p 2 — (Xj + l 2 )p 0 

dpjdt = ^p 0 + p 2 p 3 — (X 2 + p 1 )p 1 (5.3-4) 

dpjdt = X 2 p 0 + pjpa — (Xj + p 2 )p 2 

dpjdt = }. 2 p 1 + l lP2 — (pj + p 2 )p 3 

The solution of these equations requires that some 
initial conditions be specified. If an initial state of the 
system is exactly known to be S it then at t = 0 
Pi (0) = 1 and all the other initial probabilities are 
zeroes. So, for instance, the set (5.3-4) would be natural 
to solve with p 0 (0) = 1, p x (0) = p 2 (0) = p 3 (0) = 0, that 
is with both units sound at the initial moment of time. 

Linear differential equations with constant coeffici¬ 
ents may in general be solved analytically, but it is 
convenient only when the number of simultaneous 
equations is not higher than two or three. If more, the 
system is solved by numerical methods either manually 
or on a computer. 

To sum up, the Kolmogorov equations enable us to 
find the probabilities of states as functions of time. 

Answer now the question: what will be with the 
probabilities of states as t-*- oo? Will p x ( t ), p 2 ( t ), 
etc., tend to certain limits? If these limits exist and 
are independent of the initial state of the system con¬ 
cerned, then they are referred to as equilibrium proba- 
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bilities. A theorem proved in stochastic process theory 
slates that 

if the number of states n in the system is finite, and 
from each of them any other can be reached in 
a finite number of one-step transitions, then the 
equilibrium probabilities exist 6 . 

Assume that this condition is met and equilibrium 
probabilities exist such that 

lim Pi (t) = Pi (i= 1, 2, ... n) (5.3-5) 

£—►00 

The equilibrium probabilities will be denoted by the 
same symbols p u p 2 , etc. as the state probabilities, 
but one should keep in mind that these are no longer 
variables (time functions), rather they assume some 
values. Obviously, they also add to unity: 

2/0 = 1 (5.3-6) 

2=1 

Physically these probabilities are as follows. When t 
is at infinity, the system arrives at a steady state con¬ 
dition in which the system may change its states at 
random, but their probabilities do not depend on time 
any longer. The equilibrium probability of state, S t , 
may be thought of as the mean relative time the sys¬ 
tem occupies this state. To illustrate, if the system 
has three states S lt S 2 , and S 3 , and their equilibrium 
state probabilities are 0.2, 0.3, and 0.5, respectively, 
this implies that in the balanced state the system on 
the average spends two tenths of its time in S lf three 
tenths in S 2 , and a half of the time in S 3 . 

A way to compute these equilibrium probabilities is 
rather easy. If they are constant, their derivatives must 


6 This condition is sufficient but not necessary for equilibrium 
probabilities to exist. 
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be zero. Hence, to find equilibrium probabilities, the 
left-hand sides in the Kolmogorov equations should be 
set to zero to solve already the system of linear alge¬ 
braic equations. Instead of writing the differential 
equations, we might have set up the system of linear 
equations directly with reference to the graph. Trans¬ 
posing the negative term in each equation from the 
right-hand side to the left side gives the set of equa¬ 
tions having on the left-hand side the equilibrium pro¬ 
bability of a given state p t multiplied by the total 
rate of all arrivals translating from this state; and on 
the right-hand side, the multiple sum of rates of all 
arrivals translating into the ith state times the proba¬ 
bilities of the states from which they go. 

Using this rule formulate the linear equations in 
equilibrium probabilities for the system graphed in 
Fig. 5.3-2: 

(^1 + k 2 )p 0 — Fi Pi + P2P2 
iK + Hi)Pi = KPo + p*p a 
(K + = KPo + FiPs 

0*i + !**)/>» = KPi + Kp% 

This system of four equations in four unknowns might 
seem to be easily handled. The trouble, however, is 
that the equations are homogeneous and hence their 
general solution can define the unknowns only within 
an arbitrary multiplier. Fortunately, we may employ 
the normalizing condition 

Po + Pi. + pa + Pa = 1 (5.3-8) 

which makes one of the equations redundant, being a 
linear combination of the others. 

Let us solve the system (5.3-7) by specifying the 
following arrival and service rates: = 1, — 2, 
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fij = 2, p 2 = 3. Substituting the normalizing condi¬ 
tion for the fourth line in (5.3-7) yields 

3po = 2 Pi -j- 3p 2 

^Pi = Po + 3p 3 (5.3-9) 

4/> a = 2p 0 + 2 p 3 

1 — Po + Pi + Pz + Pa 

The solution to this system is 

p 0 = 6/15 = 0.40 Pl = 3/15 = 0.20 
p 2 = 4/15 » 0.27 p 3 = 2/15 » 0.13 

i.e. in the equilibrium state the system will spend on 
the average 40% of its time in S 0 (both units are heal¬ 
thy), 20% in S 1 (first unit under repair, second work¬ 
ing), 27% in S 2 (second unit under repair, first work¬ 
ing), and 13% in S a with both units under repair. 
The knowledge of these probabilities helps assessing 
the mean efficiency of the system and the load on the 
repair department. 

Suppose that the system when in state S 0 (completely 
sound) realizes profit worth 8 units per unit time, when 
in 3 units, when in S 2 5 units, and no profit when 
in S 3 . The average profit per unit time in the equi¬ 
librium state will then be 

W = 0.40 X 8 + 0.20 X 3 + 0.27 x 5 = 5.15 

We can also evaluate the utilization of repair facili¬ 
ties (load on operatives) handling both units 1 and 2. 
The fraction of time required to repair unit 1 is p t + 
+ p 3 = 0.20 + 0.13 = 0.33; the fraction of time 
needed to renew unit 2 is p 2 + p 3 = 0.40. 

At this stage we may already talk of optimization 
of the obtained solution. Suppose we are in a capacity 
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to reduce mean time of repair for one unit or the other 
(or possibly both) for a certain cost. The question is 
whether or not this reduction is worth initiating. That 
is whether or not the increase in profit will offset the 
costs incurred by the repair according to new scheme. 

We leave the formulation and solution of this econom¬ 
ic problem to the reader. Major difficulties on this 
way are four simultaneous equations in four unknowns— 
but the more precious will be the final product. 



Chapter 6 

QUEUEING OR WAITING LINE THEORY 


6.1 Objectives and Models of the Theory 

Queues of customers arriving for service of one kind or 
another arise in many different fields of activity. Busi¬ 
nesses of all types, government, industry, telephone 
exchanges, and airports, large and small—all have 
queueing problems. Many of these congestion situa¬ 
tions could benefit from OR analysis which employs to 
this purpose a variety of queueing models, referred 
to as queueing systems or simply queues. 

A queueing system involves a number of servers (or 
serving facilities) which we will call also service channels 
(in deference to the source field of the theory—telephone 
communication system). The serving channels can be 
communications links, work stations, check-out coun¬ 
ters, retailers, elevators, buses, to mention but a few. 
According to the number of servers, queueing systems 
can be of single and multichannel type. 

Customers arriving at a queueing system generally at 
random intervals of time are serviced generally for 
random times too. When a service is completed, the 
customer leaves the servicing facility rendering it 
empty and ready for a next arrival. The random nature 
of arrival and service times may be a cause of conges¬ 
tion at the input to the system at some periods when 
the incoming customers either queue up for service or 
leave the system unserved; in other periods the system 
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might not be completely busy because of the lack of 
customers, or even be idle altogether. 

A queueing system operation is a random process with 
discrete states and continuous time. The state changes 
jumpwise at the instant some events occur: an arrival 
occurs, a service is completed, or a customer unable to 
wait any longer leaves the queue. 

The subject matter of queueing theory is to build 
mathematical models which relate the specified operat¬ 
ing conditions for the system (number of channels, 
their service capacity, servicing mechanism, distribu¬ 
tion of arrivals) to the concerned characteristics of 
value—measures of effectiveness describing the ability 
of the system to handle the incoming demands. De¬ 
pending on the circumstances and the objective of the 
study, such measures may be: the expected (mean) 
number of arrivals served per unit time, the expected 
number of busy channels, the expected number of 
customers in the queue and the mean waiting time 
for service, the probability that the number in queue 
is above some limit, and so on. We do not single out 
purposely among the given operating conditions those 
intended for decision variables since they may be ei¬ 
ther of these characteristics, for example, the number 
of channels, their capacity, service mechanism, etc. 
The most important part of a study is model establish¬ 
ment (primal problem) while its optimization (inverse 
problem) is intended depending on which parameters 
are selected to work with or to change. We are not 
going to consider optimization of queueing models in 
this text with the exception made only for the simplest 
queueing situations. 

The mathematical analysis of a queueing system sim¬ 
plifies considerably when the process concerned is 
Markovian. As we already know a sufficient condition 
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for this is that all the processes changing system’s 
states (arrival intervals, service intervals) are Poisson. 
If this property does not hold, the mathematical de¬ 
scription of the process complicates substantially and 
acquires an explicit analytical form only in seldom 
cases. However, the simplest mathematics of Markov 
queues may prove of value for approximate handling 
even of those queueing problems whose arrivals are 
distributed not in a Poisson process. In many situa¬ 
tions a reasonable decision on queueing system organiza¬ 
tion suffices with an approximate model. 

All of the queueing systems have certain common 
basic characteristics. They are (a) input process ( arriv¬ 
al pattern) which may be specified by the source of 
arrivals, type of arrivals and the inter-arrival times, 
(b) service mechanism which is the duration and mode of 
service and may be characterized by the service-time 
distribution, capacity of the system, and service avail¬ 
ability, and (c) queue discipline which includes all 
other factors regarding the rules of conduct of the 
queue. 

We start illustrating the classification breakdown 
with a loss and delay system. In a purely loss system , 
customers arriving when all the servers are busy are 
denied service and are lost to the system. Examples of 
the loss system may be met in telephony: an incoming 
call arrived at an instant when all the channels are 
busy cannot be placed and leaves the exchange unserved. 
In a delay system , an arrival incoming when all 
the channels are busy does not leave the system but 
joins the queue and waits (if there is enough waiting 
room) until a server is free. These latter situations more 
often occur in applications and are of great importance 
which can be readily inferred from the name of the 
theory. 
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According to the type of the source supplying custom¬ 
ers to the system, the models are divided into those 
of a finite population size , when the customers are only 
a few, and the infinite-population systems. The length 
of the queue is a subject to further limitations imposed 
by allowable waiting time or handling of impatient cus¬ 
tomers which are liable to be lost to the system. 

The queue discipline, that is the rule followed by 
the server in taking the customers in service, may be 
according to such self-explanatory principles as 
“first-come, first-served”, “last-come, first-served”, or 
“random selection for service”. In some situations 
priority disciplines need be introduced to allow for 
realistic queues with high priority arrivals. To illus¬ 
trate, in extreme cases the server may stop the ser¬ 
vice of a customer of lower priority in order to deal 
with a customer of high priority; this is called pre¬ 
emptive priority. For example a gantry crane working 
on a container ship may stop the unloading half-way 
and shift to another wharf to unload perishable goods 
of a later arrived ship. The situation when a service 
of a low priority customer started prior to the arrival 
of a high priority customer is completed and the high 
priority customer receives only a better position in the 
queue is called non-preemptive priority. This situation 
can be exemplified by an airplane which enters a queue 
of a few other aircraft circling around an airport and 
asks a permission for an emergency landing; the 
ground control issues the permission on the condition 
that it lands next to the airplane being on the runway 
at the moment. 

Turning over to the service mechanism, we may find 
systems whose servicing channels are placed in parallel 
or in series. When in series, a customer leaving a pre¬ 
vious server enters a queue for the next channel in 
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the sequence. For example, a workpiece being through 
the operations with one robot on a conveyor is stacked 
to wait when the next robot in the process is free to 
handle it. These operation stages of a series-channel 
queueing system are called phases. 

The arrival pattern may and may not correlate with 
the other aspects of the system. Accordingly, the sys¬ 
tems can be loosely divided into “open” and “close”. 
In an open system, the distribution of arrivals does not 
depend on the status of the system, say on how many 
channels are busy. To contrast, in a close system, it 
does. For example, if a single operative tends a few 
similar machines each of which has a chance of stopping, 
i.e. arriving for service, at random, then the arrival 
rate of stoppings depends on how many machines have 
been already adjusted and put on or those not yet 
serviced. The classification examples might, of course, 
be added, but we constrain ourselves with those 
considered. 

An optimization of a queueing system may be at¬ 
tempted from either of two standpoints, the first in favor 
of “queuers” or owners of the queue, and the second 
to favor the “queuers”, i.e. the customers. The first 
stand makes a point of the efficiency of the system and 
would tend to load all the channels as high as possible, 
i.e. to cut down idle times. The customers on the 
contrary would like to cut down waiting time in a 
queue. Therefore, any optimization of a congestion neces¬ 
sitates a “systems approach” with the intrinsic com¬ 
plex evaluation and assessment of all consequences for 
each possible decision. The need for optimality over 
conflicting requirements may be illustrated with the 
viewpoint of the customer wishing to increase the 
number of channels which, however, would increase 
the total servicing cost. The development of a reason^ 
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able model may help solving the optimization prob¬ 
lem by choosing the number of channels with account 
for all pros and cons. This is the reason why we do 
not suggest a single measure of effectiveness for all 
queueing problems, formulating them instead as mul¬ 
tiple objective problems. 

All the mentioned forms of queues (and many others 
for which we give no room here) are being studied by 
queueing theory where there is a huge literature on the 
subject. We quote only a few books [13-17]. As a rule, 
the topics dealing with the theory can be found in 
texts on operations research [1, 6, 71. The discussion, 
though, almost nowhere tunes an appropriate method¬ 
ological level: the derivations are often too complicated, 
and the formulas are deduced not in the very best 
way. 1 In this (for the lack of room) brief discussion of 
queueing theory we are going to display two method¬ 
ological approaches which substantially facilitate the 
process of deriving some valuable formulas. These 
approaches are the focus of the subsequent section. 


6.2 The Birth and Death Process. 

The Little Formula 

The Birth and Death Process. As will be recalled, 
with a directed graph at hand we can easily derive 
the Kolmogorov equations for probabilities of states 
to covert them to, and solve, the algebraic equations 
in the equilibrium state probabilities. In certain cases 
the latter can be solved beforehand in a general form. 
One of the special cases when it can be succeeded real- 


1 The author ought to confess, however, that her own 
book [ 6 ] unfortunately suffers from the same setback—lack 
pf simplicity in certain derivations. 
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izes with the states of the considered system forming 
the so-called ‘birth and death process’. 

An illustrative graph for the birth and death pro¬ 
cess is shown in Fig. 6.2-1. This graph is outstanding 
in that all its vertices form a single chain and are 


J-o( _ An _ A 23 A-k- 1,k Ak'k+f _ A n -in 


A/, 


l 21 *32 *k,k-l 

Fig. 6.2-1 


l n,n-t 


each (except the initial and terminal vertices, S 0 
and S n ) joined with each of the adjacent vertices by 
two (direct and reverse) arcs. The term ‘birth and death 
process’ is due to biological problems where it de¬ 
scribes populational variations. 

The birth and death model is often exploited in 
various applications, in particular in queueing theory, 
therefore it would be useful to derive the expressions 
for its equilibrium probabilities. 

Assume that all the processes translating the system 
through the arcs of the graph are Poisson. Referring to 
the graph in Fig. 6.2-1, set up and solve the algebraic 
equations for the steady-state probabilities (their 
existence stems from the fact that each state is reach¬ 
able for any other and the number of states is finite). 
For the first state, we have 

^oiPo = ^ioPt (6-2-1) 

For the second state, S t : 

(^12 ~t~ ^lo)Pl = ^ OlPf) ~t" ^2lP2 
which, in view of (6.2-1) reduces to 
^uPi “ ^2iP 2 
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and further in exactly the same manner 

^ 23^2 = ^ 32^3 

so that in general 

^h-l.h Ph -1 = ^ h.k-lPk 

where /c assumes all integer values from 0 to re. Being 
collected together, the equilibrium probabilities p 0 , 
p 1 , . . p n satisfy the system 

^oiPo — ^ 10 P 1 
^lzPl = ^2lP2 


-1,J< Ph-l — Ph 


( 6 . 2 - 2 ) 


-l.nPn-1 ^ n.n-'lPn 

which, besides, must be augmented by the normalizing 
condition 

Po + Pi + Pi + • • • + Pn = 1 (6.2-3) 


To solve the resulting set of equations, express in the 
first line of (6.2-2) p x via p 0 : 

Pi = T^Po (6.2-4) 

Mo 

The second line by account of (6.2-4) yields 


(6.2-5) 


and the third line, subject to (6.2-5), gives 
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or, for any k (from 1 to n): 

X-k-l, k ■ • ■ ^ 12^01 

Xh , fc-l' ■ ■ ■ ^21^10 


Ph 


P 0 


(6.2-7) 


We would like to attract attention to formula (6.2-7). 
Its numerator is the multiple of all rates attending all 
arcs leading from left to right, from the origin to a given 
state S h , and the denominator contains the product 
of all rates attending the arcs leading from right to 
left (from to the origin). 

Thus all the probabilities of states p 0 , p l5 . . .,_ p n 
are expressed in terms of one of them, viz. p 0 . Substi¬ 
tuting these expressions into the normalizing condi¬ 
tion (6.2-3) and factoring out p 0 yields 



^12 ^01 
X 2 i Vio 


+ ... + 


Xn- 1 , n 
X n, n-i 


^12^01 

X 21 X 10 



or, solving for p n : 


P o = 



^12 ^01 
X 2 1 ^10 


. . . + 


^n-i, n 
Xn, n-i 


..X 

..X 


12^01 i 1 
21^10 ' 
( 6 . 2 - 8 ) 


Note that the coefficients of p 0 in each of the formu¬ 
las (6.2-4)-(6.2-7) are the sequential terms in the series 
following the unity in (6.2-8). Hence in computing p 0 
we have found all these coefficients. 

The derived formulas are of great value in solving 
simplest problems of queueing theory. 

The Little Formula. Now we denote an important 
formula relating (for a settled queue, i.e. that in steady 
state) the expected number in system, L, i.e. the expected 
number of customers either waiting in line and/or being 
serviced, and the expected system time, W, which is the 
time a customer spends waiting plus being serviced. 

In a queueing system, vhich can be either of the fol- 
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lowing: single or multichannel, Markov or non-Mar¬ 
kov, with unlimited or finite size of queue, we will 
consider two processes: that of the customers arriving 
at the system and that of being serviced and leaving it. 


x(o m 



Fig. 6.2-2 

If the queue is settled and the system is in the steady 
state, the mean number of arrivals per unit time (input) 
is equal to the number of elements leaving it (output), 
both the input and the output have the same rate X. 

Let X ( t) denote the number of customers arriving 
at the system up to time t, and Y ( t ) the number of cus¬ 
tomers leaving it by the same instant t. Both functions 
are random and vary in jumps of unit value at the 
instants an arrival occurs, X ( t ), or leaves the system, 
Y ( t ). The forms of both X ( t ) and Y ( t ) are shown in 
Fig. 6.2-2; both lines vary in steps, the upper is 
X ( t ), the lower is Y ( t ). Obviously, for any moment of 
time t their difference Z (t) — X (£) — Y ( t) is but 
the number of customers in system. When the lines 
X {t) and Y ( t) merge, the system has no customers. 

Consider now what happens in the long run, taking 
a large period of time T (we can mentally extend the 
graph far beyond the margins of the page) and comput¬ 
ing for it the expected number in the system. It will 
be given by the integral of the function Z (£) over this 
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period divided (in order to average) by the period T 

T 

L = ±-^Z(t)dt (6.2-9) 

o 

This integral can be identified as the hatched area 
in Fig. 6.2-2. Let us consider this figure in detail. The 
plot sums up of rectangles each of unit altitude and a 
base equal to the time the respective customer spends 
in system. We shall index the customers time in sys¬ 
tem as t u t 2 , etc. Closer to the end of the period T 
some rectangles will not completely fit the hatched 
area, but at a sufficiently large T this will not matter. 
Hence we may safely equate 

T 

jz(t)df =]>]*, (6.2-10) 

0 > 

where the summation was performed over all the ar¬ 
rivals for time T. 

Dividing both sides of (6.2-10) by the length of the 
period T we get in view of (6.2-9): 

i = (6.2-11) 

i 

Dividing and multiplying the right-hand side of 
(6.2-11) by the rate X yields 

i 

In the quantity TX we can recognize the expected 
number of arrivals for the period T. If we divide the 
sum of all the times t t by the expected number of cus¬ 
tomers, we obtain the expected time of a customer in 
system, W. Therefore 

L = XW 
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or 

W = L/l (6.2-12) 

This result is known as Little's formula which states 
for any distribution of arrival and service times, and 
any service mechanism that 

the expected system time is equal to the expected 
number of customers in system divided by the arri¬ 
val rate. 

In the same manner we could derive a second Lit¬ 
tle’s formula relating the expected time in queue , W q , 
which is the expected time a customer spends waiting 
in line, and the expected number in queue, L q , which is 
expected number of customers waiting to be serviced: 

W q = L q /X (6.2-13) 

To effect the derivation, it suffices to use as the bot¬ 
tom line in Fig. 6.2-2 the function U ( t ), the number 
of customers that leave the queue rather than the sys¬ 
tem by time t (if a customer entering the system im¬ 
mediately goes to service missing the queue, we may 
still assume the arrival joins the queue but waits in 
it a zero time). 

The Little formulas (6.2-12) and (6.2-13) are signi¬ 
ficant results of queueing theory. Unfortunately, most 
texts on the subject ignore these formulas (proved in 
the general form relatively recently). 2 

6.3 Analysis of Simplest Queueing Models 

This section will focus on some simplest queueing sit¬ 
uations and their measures of effectiveness. Deducing 
the respective formulas we will demonstrate the main 
techniques characteristic of this part of the theory deal- 

2 The popular text [18] presents a different development of 
Little’s formula as well as an easy introduction to the theory. 
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ing with Poisson processes. We will not aim at il¬ 
lustrating as many models as possible; this text indeed 
is not a handbook of operations research (as such we 
may point to [32]). Instead we intend to make the read¬ 
er familiar with those “little ingenuities” which fa¬ 
cilitate the investigation of the theory so that it does 
not seem a haphazard collection of examples. 

In this section we will assume that all the inter¬ 
arrival and service times are in a Poisson process. 
The servicing process will be understood as a sequence 
of intervals resulting for the customers being served by 
a single continuously busy channel. About this process 
we may say that it has an exponential distribution or 
that the service-times are (negative) exponentially 
distributed, so we will use the names interchangeably. 
The required effectiveness measures for the considered 
queues will be introduced in the cause of discussion. 

(i) Multichannel queue having no waiting room 
(Erlang loss system). This model of n servers and no 
waiting room for any n customers is one of the first, now 
classical problems in queueing theory. It was posed by 
telephone communications needs and solved at the verge 
of the century (1909) by A. K. Erlang for the Copen¬ 
hagen Telephone Company. It is formulated as fol¬ 
lows: n channels (trunklines) handle the demands (calls) 
incoming at arrival rate X. Service (connection) is 
being provided at service rate p, which is the recipro¬ 
cal of the mean service time. It is assumed that the 
calls that cannot be supplied with lines are lost. It is 
required to find the stationary probabilities of the 
states in the queue and evaluate its measures of effec¬ 
tiveness: 

A = absolute throughput (channels utilization) 
which is the average number of demands 
served per unit time in the system, 
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Q = relative throughput which is the average pro¬ 
portion of arrivals served in the system, 
Pj 0S1 = probability of loss which is the probability 
that an arrival leaves the queue unserved, 
k — average number of busy channels. 

SOLUTION. We will number the states of the system 
in accordance with the number of demands being in 
the system (in the circumstances it coincides with the 
number of busy channels) as follows: 

S 0 = no arrivals 

S x = one arrival in the system (one channel is bu¬ 
sy, the others are idle) 


S h = k demands in the system (k channels are busy, 
the others free) 


S n = n demands in the system (n channels busy). 
The graph for this system corresponds to that of 
the birth and death process. Fig. 6.3-1 shows the di- 



Fig. 6.3-i 

graph with the corresponding rates indicated at the 
arcs. The transitions from S 0 to S t occur at the rate 
X (as soon as a demand arrives the system moves from 
S 0 to S j). Generally, the system’s transitions from each 
lower (left on the digraph) state to the next (right) state 
proceed at the arrival rate X (the upper arcs of the 
digraph). 

Consider now the rates at the lower arcs in the digraph. 
Let the system be in S lt i.e. with one channel busy 
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servicing. It is getting service to p demands per unit 
time. Hence the service rate p at the SxSq arc. Now 
imagine that the system is in S 2 with two channels 
busy. To move in S t it needs that either the first or 
the second channel completes its service; the total 
rate of servicing rendered by these channels is 2p; it 
is placed at the S 2 S 1 arc. The total service rate of 
three channels is 3p, and of k channels Ap. These rates 
are indicated below the bottom arcs in the digraph. 

With all these rates at hand we can now make use 
of the expressions (6.2-7) and (6.2-8) for equilibrium 
probabilities in the birth and death process. From 
(6.2-8) we have 

Po ~ 1 1 + p + 2p 2 2-3p 2 + • • • 

I A^ . f A" \ ~ ^ /n q 4 \ 

+ k\ p* + •** + ra! p» ) (6.3-1) 


The terms of the series in the parentheses, A./p, 
A 2 /2p 2 , ... X n !n\ p", are the coefficients of p 0 in the equi¬ 
librium state formulas for p lt p 2 , ..., p n : 


jP* —JT Po ' Pz ~ 2p 2 ' Po ' ‘ ’ 

Xh _ A" 

Ph Alp* Po ' •••’ Pn nip" Po 


(6.3-2) 


Notice that the rates A and p enter the expressions 
(6.3-1) and (6.3-2) only as the ratio A/p which is com¬ 
monly denoted as 

p = A/p (6.3-3) 


and called traffic intensity being the ratio of effective 
arrival and service rates. Essentially it is the mean num¬ 
ber of customers arriving for the time of an arrival 
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being served. Rewrite the last formulas in terms of 
this symbol: 

Po=(l + P + fj-+ • • . +if+ • • • +-£f) (6.3-4) 

P 2 

Pi PPo» P 2 . — * 2 |* Po> • • • > 

Pfc—•••. Pn^-^Po (6.3-5) 

These expressions give the equilibrium probabili¬ 
ties of states in the system. We will employ them to 
deduce the performance measures of the system. First, 
we evaluate the probability P i 08S that an arrival will 
not find a free channel and be lost to the system. This 
condition* is present when all n channels are busy, 
whence 

^losi = Pn ~ Po (6.3-6) 

This formula is often called Erlang's loss formula. 
It gives the proportion of time for which the system is 
fully occupied and in the telephone application, with 
randomly arriving calls, this is the proportion of lost 
calls. We may thus find the probability that a demand 
will be served, i.e. the relative throughput: 

e=l--Pio..= l — J (6.3-7) 

Multiplying Q by the arrival rate k yields the abso¬ 
lute throughput 

= ^ = g-p 0 ) (6.3-8) 

Now we need only to find the mean number of busy 
channels k. We can do it directly as the mathematical 
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expectation of a discrete random variable assuming 
the integer values in the range from 0 to n with the 
respective probabilities p 0 , p t , ..., p n , namely 

k = 0-/? o + 1 •p 1 + 2 -p 2 + ... + K‘P n 

Having substituted the respective expressions for 
Ph (& = 0, 1, ..., n) from (6.3-5), after some manipu¬ 
lation we could finally arrive at a correct formula for 
k. But we shall demonstrate a more simple way to it 
by means of a promised “little ingenuity”. Indeed, we 
know the absolute throughput A. This quantity, how¬ 
ever, is nothing but the rate of served customers leav¬ 
ing the system. Each busy channel serves on the 
average p customers per unit time. Hence the mean 
number of busy channels is 

k = Alp, (6.3-9) 

or substituting for A from (6.3-8) and recalling that 
kip — p 

* = p ( 1 ~~'7r Po ) (6.3-10) 

We recommend the reader to solve a self-check ex¬ 
ample. An exchange possesses three trunk lines (n = 3), 
the arrival rate of calls put for connection is k = 1.5 
(calls per minute); the mean time to serve a call is 2 
(minutes). Assuming a Poisson distribution of arrival 
and service times, find the steady-state probabilities 
and the performance measures, A, Q, P loss , and k. 
For the sake of comparison, the answer must be this: 
p 0 = 1/13, px — 3/13, p 2 — 9/26, p 3 = 9/26 « 0.346, 
A « 0.981, Q 0.654, Pi 0 ss~ 0.346, and k « 1.96. 

These results indicate, by the way, that our exchange 
is substantially overloaded: of three channels about 
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two are busy in the mean, and about 35% of all in¬ 
coming calls leave unserved. We suggest the reader 
evaluates how many channels (trunk lines) the exchange 
should possess to supply not less than 80% of sub¬ 
scribers with a required line. What proportion of chan¬ 
nels will then be idle? 

Here there are already some trends to optimization. 
To illustrate, each channel has a cost per unit time. On 
the other hand, each served call realizes a certain profit. 
Multiplying this profit by the average number of calls 
served per unit time, A, gives the mean profit rate from 
the system. Naturally this profit would increase with 
any new trunk line, but the maintenance costs of the 
lines also increase. Which will outbalance, the rise 
of profit or expenses? It depends on the conditions of 
the operation, i.e. the monetary rate per served call 
and the maintenance cost of a channel. With these 
values specified, one can find an optimal number of 
lines which would be most efficient from the economic 
viewpoint. We leave these computations to the reader, 
who is to invent and solve his own example. The inven¬ 
tion of problems generally gives more intellectually 
than the solution of those posed by somebody. 

(ii) Single channel queue of unlimited size. Single¬ 
server queues are most often met in applicational areas. 
The examples are a queue to a doctor, a line of foot¬ 
ball fans to a ticket seller, an on-line computer perform¬ 
ing the subscriber demands in the time sharing mode, 
etc. In queueing theory these single-server models are 
also of special importance (most analytical expressions 
derived so far relate to these systems). Therefore we 
shall attach an especial attention to this type of queue. 

Let there be a single-server queue with no constraints 
imposed either on the queue length or waiting time. 
The customers arriving at a rate K are serviced at a 
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rate p. We will seek for the equilibrium probabilities 
of the queue size and the performance measures: 
L — expected number of customers in the system, 
W = expected system time, 

L q — expected number of customers in the queue, 
W q = expected time in queue, 

P = probability that the server is busy 
(server utilization factor). 

We have no need to calculate the throughputs A 
and Q since the queue has ample waiting room and each 



Fig. 6.3-2 


customer sooner or later will be served, therefore 
A — X, and Q = 1. 

SOLUTION. We shall index the states of the system 
as before according to the number of customers in the 
system, viz. 

S 0 — the server is free, 

Sy = the server is busy (serves an arrival), no 
queue, 

S 2 = the server is busy, one customer in the queue, 


S h = the server is busy, k — 1 customers wait 
in line, 


Theoretically the number of states in this system 
is infinite. The digraph of the system is displayed in 
Fig. 6.3-2. It corresponds to the birth and death pro¬ 
cess with a countably infinite number of states. The 
arrival rate k moves the system from the left vertex 
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to the subsequent vertex at the right, and the service 
rate p is responsible for the transitions in the oppo¬ 
site direction. 

First of all we should ask ourselves whether station¬ 
ary probabilities exist for this queue. The number of 
states is indeed infinite and principally, as t-*~ oo, 
the queue can increase without limit. It indeed can 
take place: the equilibrium probabilities for the sys¬ 
tem exist only when it is not overloaded. It can be 
shown that these probabilities exist when p < 1, oth¬ 
erwise, at 1, the queue increases without bound 
as t tends to infinity. This fact seems intuitively strange 
for p = 1. The system might seem to have no im¬ 
possible requirements imposed, namely one customer 
arrives in the mean during a single service, so the con¬ 
ditions are for proper operation, still it does not work. 
An explanation is that at p = 1 the system is able to 
handle the arrivals only if they are regular along with 
the service intervals which are, due to unity traffic 
intensity, equal to the intervals between the succes¬ 
sive arrivals. In this “ideal” situation, no queue will 
form, the channel will be continuously busy serving 
the arriving customers at a regular basis. Should, 
however, the arrival or service rate acquire a bit of 
randomness, the queue length will increase infinitely. 
We have no such situation in practice, though, be¬ 
cause an infinite queue is an abstraction. The cost of 
using mathematical expectations in place of random 
variables may as is seen be rather high! 

Now switch back to our single-channel queue with 
an infinite source population. Strictly speaking, the 
formulas for equilibrium probabilities in the birth 
and death process were derived only for the case of fi¬ 
nite number of states, however, we will loosely apply 
them to the infinite state situation. Compute the 
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steady-state probabilities of the states with formulas 
(6.2-8) and (6.2-7). In our case the number of addends 
in (6.2-8) will be infinite. Thus the expression for p 0 
will be 

Po = ~f~ (X/p ) 2 + ••• -f- (^/p) fe "f" 

= (1 + P + P 2 + + P k + -)- 1 (6.3-11) 

The series in the parentheses is a geometrical pro¬ 
gression. As will be recalled, it converges at p < 1 
and diverges at p ^ 1. This is though not a strong, still 
an implicit proof that the equilibrium probabilities 
Po, p lt p 2 , ..., Phi • •• do exist only for p < 1. Summing 
up the series in (6.3-11) yields 

l + P + P 2 +...+P ft +... = T ^- 
or for (6.3-11) 

Po = 1 - P (6-3-12) 

The equilibrium probabilities are then found with 
the expressions 

Pi = PPo. Pi = P 2 Po> —»Pft = P*Po»— 
which, by virtue of (6.3-12) become 

Pi = P (1 - p), Pa = p 2 (1 — p),..., 
p h — p s (1 — p), ... (6.3-13) 

The probabilities p„, p lt ..., p h are seen to form a 
geometric series with the common ratio p. However 
strange it might seem, the largest of them is p 0 , that 
is the probability of idle channel. In other words, 
whatever the load of the server, if it is capable of serv¬ 
ing the incoming arrivals (p < 1), the most proba¬ 
ble number of customers in the system will be zero. 
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Obtain the average number of customers in the 
sytsem, L. This will be a cumbersome operation. The 
number of customers in the system is a random vari¬ 
able. It assumes integer values 0, 1, 2, ..., k, ... with 
probabilities p 0 , p t , p 2 , ..., • ••, respectively. Its 

mathematical expectation is 

oo 

L = 0* Po+ Pi+ 2* Pz~\~ • • • + 'k' Ph "f •••= S kph 

ft=l 

(6.3-14) 

where the summation starts from 1 rather than 0 
because the first term is zero. 

Substituting in (6.3-14) the expression for p h from 
(6.3-13) gives 

L= s V(i-p) 

ft=l 

or with p (1 — p) factored out ahead of the sum 
£=P(1-P) 2 V" 1 

/:=i 

We may again exercise our ingenuity if we identify 
that &p ft-1 is but a derivative of p h with respect to p, 
hence 

i=p<‘-p)i 

h= 1 

Interchanging the places of the differential and summa¬ 
tion operators, we get 

OO 

i-pO-rt^-S p' 

h =1 


(6.3-15) 



6.3 Analysis of Simplest Queueing Models 


185 


The sum in (6.3-15) can be recognized as a converg¬ 
ing geometric series with both the first term and the 
common ratio being equal to p; it totals to p/(l — p) 
and its derivative is (1 — p) -2 . Substituting this de¬ 
rivative in (6.3-15) yields 

£ = (6.3-16) 

Now we may invoke Little’s formula (6.2-12) and 
find the mean time an arrival spends in the system: 


W = 


P 

Ml-P) 


(6.3-17) 


To find the mean number of customers in the queue, 
L q , we will think this way: the number of customers 
in the queue is that in the system minus the custom¬ 
ers being served. Hence (according to the rule of 
adding the expectations) the expected number of cus¬ 
tomers in the queue, L q , is the expected number in 
the system L, less the expected number in service. 
The number of customers getting service may be either 
zero (server is idle) or unity (server is busy). The 
mathematical expectation of such a random variable 
equals the probability that the server is busy, P. Ob¬ 
viously, P equals unity minus the probability p 0 
that the server is free: 

P = 1 - Po = p (6.3-18) 


Consequently, the mean number of customers being 
served is 


whence 


L s P 


(6.3-19) 
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or, finally, 

£* = -£p (6-3-20) 


Referring to the Little formula (6.2-13) find the ex¬ 
pected time in the queue 


W q 


P 2 

Mi-P) 


(6.3-21) 


By this we have found all the sought performance 
measures of the system. 

As a self work we suggest that the reader should 
solve the following example. A single-channel queueing 
system is a railway sorting yard which handles a Poi¬ 
sson arrival of trains incoming with rate X = 2 
(trains per hour). A gathering of a train (servicing) takes 
a random time exponentially distributed with a mean 
of 20 minutes. The yard has waiting room for only two 
trains to be positioned on two body tracks; if both 
of them have waiting trains, the arriving trains have to 
await service on the approach lines. It is required to 
find (for the yard operating at the equilibrium condi¬ 
tions): 


L — mean number of trains bound to the yard, 
W = mean time a train spends in the yard (on 
the body tracks, on the approach lines, 
and in service), 

L q — mean number of trains waiting in queue 
for gathering (no matter on what track), 
W q = mean time a train waits in the queue. 

Try also to evaluate: 


L a = mean number of trains awaiting at the 
approach lines, and 
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W a = mean time of waiting on the approach 
lines (hint: this quantity is related to the 
previous by Little’s formula). 

Last but not least, find the total diurnal penalty 
which the yard would have to pay off for the detention 
of trains on the approach lines given that a train delay¬ 
ed on these lines by one hour incurs a penalty rate 
worth a monetary units. 

To check up, we disclose the answer: L = 2 (trains), 
W = 1 (hour), L q — 4/3 (trains), W q = 2/3 (hour), 
Lg. — 16/27 (train), W n = 8/27 « 0.297 (hour). The total 
diurnal penalty results by multiplying the expected 
number of trains arriving at the sorting yard per day, 
the expected waiting time on the approach lines, and 
the hour penalty rate a; for our example it is « 14.2a. 

(iii) «-server queueing system with unlimited queue 
size. This model is developed similar to the previous 
for the single-channel queue, though the procedure 
this time will be a bit more involved. We again will 
index the states in accord with the number of custom¬ 
ers in the system: 

S 0 = no arrival (all the channels are free), 
= one channel is busy, the others are idle, 
S 2 = two channels are busy, the others are 
idle, 


S h = k channels are busy, the others are idle, 


S n = all n channels are busy (no queue), 

S n+1 = all n channels are busy, one customer 
waits in line, 


S n+r = all n channels are busy, r customers in 
the queue, 









188 


Ch. 6 Queueing or Waiting Line Theory 


The digraph for the system is depicted in Fig. 6.3-3. 
We leave for the reader to think over and substantiate 
the rates shown next to the arcs of the graph. It is a 
birth and death process with an infinite number of 
states. We give without proof the natural condition 



Fig. 6.3-3 


for the equilibrium state probabilities to exist: 
p In C 1. If otherwise, p/n ^ 1, the queue grows with¬ 
out bound. 

Suppose the condition p/n < 1 is satisfied and the 
equilibrium distribution of probabilities exists. Ap¬ 
plying the very same formulas (6.2-8) and (6.2-7) 
of the birth and death model, we will find these bal¬ 
ance probabilities. The expression for p 0 will contain 
certain terms with factorials and the sum of an infi¬ 
nite geometric sequence with the ratio p/n: 


A> = [ 1 +-£- + -|r + • • • 



p n+1 -|-i 

nl (n — p) J 


Pi — P o> 



_ P n 

PO' • • • > Pn n j P 0 

(6.3-22) 


Pn+l 


n-nl 


P 0> 


Pn+r — 


n r - rel 


Po> 


Now we will seek for the performance measures of 
the system. The easiest to find is the average number 
of busy channels k = X/p = p (this is in general val- 
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id for any queue of unlimited length). Next in turn 
are the average number of customers in the system, 
L, and the average number in the queue, L q . We ap¬ 
proach firstly L q , which is easier, by the expression 

oo 

L q 2 TPn+r 

i=l 


performing the required manipulations as in the deri¬ 
vation of queueing model (ii) (with differentiation of 
the series) to obtain 


Lq 


P n+l Po 

n-n\ (1—p/n) 2 


(0.3-23) 


Adding up the average number of customers in ser¬ 
vice (which is the same as the number of busy servers) 
k = p gives 

L = Lq + p (6.3-24) 

Dividing the expressions for L and L q by X, in ac¬ 
cordance with Little’s formula, leads to the mean 
times a customer spends in the system and in the queue 

W = LIX 

W q = Lqlk (6.3-25) 

Now we solve an interesting example. A booking 
office with two clerks sells tickets for two destinations 
A and B. This queueing system is a two-server queue 
with unlimited length. The passengers bound for ei¬ 
ther A or B arrive at an equal rate of X A = k B = 
= 0.45 (passengers per minute) to build up a total ar¬ 
rival with the rate k A + = 0.9. It takes a clerk 

two minutes, on the average, to serve a passenger. 
From the previous operation the service is known to 
produce queues of complaining passengers. The man- 
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agement is contemplating to divide the present of¬ 
fice into two specialized ones, selling tickets one only 
for A, the other only for B. We are to verify by compu¬ 
tation whether this decision is reasonable or not. Since 
we know only how to evaluate the characteristics 
of Poisson queues, we assume that both arrivals and 
service processes are Poisson (this assumption will 
not impede the qualitative side of the analysis). 

To compare, consider two modes of booking-office 
operation, the existing and the suggested one. 

OPTION I (existing). The two-channel queueing 
syslem handles the customers arriving at the rate 
k = 0.9 with the service rate p, = 0.5, so that p = 
= X/p, = 1.8. Since p/2 = 0.9 < 1, the equilibrium 
state distribution exists. With the first formula in 
(6.3-22) we find p 0 & 0.0525. The average number of 
customers in the queue is found by (6.3-23), viz. 
L q x 7.68; the mean time in the queue (by the first 
formula in (6.3-25)) W q = 8.54 (min). 

OPTION II (suggested). Here we are to consider 
two single-server queues of the arrival rate k — 0.45 
and of previous p, = 0.5. Since p — kl p. = 0.9 < 1, 
the equilibrium distribution exists. From (6.3-20), 
the expected queue size (of one clerk) L q = 8.1. 

The result is entirely unexpected. The queue grows 
more instead of being shorter. Perhaps the mean wait¬ 
ing time has been cut down? Let’s have a glance. 
Dividing L q by k — 0.45, we get W q « 18 (minutes). 

Just on the opposite! Instead of being relaxed, both 
the mean queue size and the mean waiting time in it 
have risen up. 

Can we guess why it happened this way? Upon a 
little thought we might conclude that this was so be¬ 
cause in option I (two-server queue) the average pro¬ 
portion of time each of the clerks was idle is shorter. 
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Indeed, here a clerk completing a service of a custom¬ 
er may begin servicing either a passenger bound for 
B or a passenger bound for A. In option II, a clerk 
being devoid of this interchangeability, should his 
own queue vanish, would idly look at the passengers 
queueing up towards his busy servicing mate. 

The reader might, well agree with the explanation 
of the increase; but why is it that substantial? Can it 
not be due to a computational error? There was no er¬ 
ror, yet the point is that both queues operate on the 
verge of their possibilities; should the service time 
increase (p, diminish) slightly, the servers would lose 
their control on the queue which would grow without 
bound. The idle time of a server is in a sense equiva¬ 
lent to a decrease in productivity. 

So the result that seems paradoxical at first glance 
(or simply wrong) proves to be correct and fully explain¬ 
able. Queueing theory is rich in paradoxical results 
of the kind whose cause is not plainly evident. The 
author herself would repeatedly get “surprising” re¬ 
sults to be proved perfectly correct later. 

In an attempt to drive the above situation to a posi¬ 
tive end, the reader may argue that if a clerk sells 
tickets for one destination only, the service time must 
be cut down at least somewhat if not by half (recall 
that in the computation we have assumed an unchanged 
figure of 2 minutes for the mean service time). We sug¬ 
gest that such an astute reader should answer in what 
proportion this time must be decreased to receive a 
positive economic effect from the two-booking-offices 
decision. In this formulation we again encounter, 
though elementary, an optimization problem. Even 
with the simplest Markovian models the preparative 
computations help elucidating the concerned pheno¬ 
menon qualitatively: which option is profitable and 
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which is not. The next section introduces some simple 
non-Markov queueing models which would widen the 
scope of our possibilities in the area. 

Now that we hope the reader is familiar with the 
techniques of equilibrium probabilities evaluation 
and with the computations of performance measures 
for the simplest queueing systems (the birth and death 
model and Little’s formula), the family of discussed 
queues can be augmented by two more Poisson sys¬ 
tems suggested for self-testing. 

(iv) Single-server queueing system with limited 
waiting room. 

This model differs from that with unlimited queue 
size only in that the number of customers in the queue 
cannot go beyond some specified length m. If an arri¬ 
val occurs when m customers are awaiting in the queue 
already, it leaves the system unserved (is lost to the 
system). To characterize the model one needs to eval¬ 
uate besides the equilibrium probabilities (by the 
way, in this model they exist for any p because the 
number of customers in the queue is finite), the proba¬ 
bility of being lost to the system, Pioss> the absolute 
throughput, A , the probability that the channel is 
busy, P, the mean length of the queue, L q , the expect¬ 
ed number of customers in the system, L, the expect¬ 
ed waiting time in the queue, W q , and the mean sy¬ 
stem time, W. In deriving the performance measures 
one may employ the same technique as for model (ii) 
with the difference that this time a finite rather than 
infinite progression must be summed up. 

(v) Machine servicing model. Another name for this 
system in queueing theory literature is the machine 
interference model. It refers to the situation in which 
a server (or a team of servers) maintains a group of m 
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machines. Each machine is in one of two states: ei¬ 
ther “up” (running) or “down” (requiring repair service). 
The demands for servicing arrive at a rate X. If the 
server is free, when a machine breaks down, he imme¬ 
diately begins the service of the machine. If a machine 
breaks down when the server is busy, it joins the queue 
and waits until the server is free. The mean time to 
serve a machine is 1/p. The arrival rate of “demands” 
for attendance incoming from stopped machines to 
the server depends on how many machines are oper¬ 
ating. If k machines are running, then the arrival rate 
is kX. To characterize the system one needs to evaluate 
the equilibrium probabilities of the states in the 
system, the mean number of running machines, and the 
probability that the server will be busy. 

It is interesting to observe that in this queueing 
system the equilibrium state probabilities will exist 
at any values of % and p, because the number of states 
in the system is finite. 


6.4 More Complex Queueing Models 

This section will give a brief discussion on some non- 
Poisson queueing systems. Up to now all formulas 
have been derived, or at least could be derived, by the 
reader in command of the birth and death process and 
differentiation techniques. The results of this section, 
however, the reader will have to accept without proof. 

So far we have dealt with only the simplest queueing 
systems for which all the processes responsible for the 
transitions from state to state within the system were 
Poisson. What will we do if they are not? How close 
is the assumption of the Poisson distribution models to 
what we have in reality? IIow significant are the errors 
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involved when its fit to the reality is not perfect? In 
what follows we shall attempt to answer these ques¬ 
tions. 

Unfortunately we have to openly confess that non- 
Markov queueing theory has no significant achievements. 
For non-Markov queueing systems, there exist only a 
few developments which enable the queue performance 
measures to be expressed in terms of the commonly 
specified conditions: number of servers, distribution of 
arrivals, and distribution of service times. Below we 
discuss some of these results. 

(i) Multichannel queueing system with no waiting 
room, with Poisson arrivals and an arbitrary distribu¬ 
tion of service times. In the previous section we de¬ 
duced the Erlang formulas (6.3-4) and (6.3-5) for the equi¬ 
librium probability distribution in the loss multiserv¬ 
er queueing system. Of a comparatively recent origin 
(1959) is the work by Sevastyanov [19] who proved 
that these formulas are valid not only for an exponen¬ 
tial but also for an arbitrary distribution of service 
times. 

(ii) Single-server queue with unlimited queue size, 
Poisson arrivals, and arbitrary distribution of service 
times. If a single-server queueing system with an un¬ 
limited queue size has a Poisson input of rate X, and 
service times are arbitrarily distributed with mean 
1/p, and coefficient of variation v^, then the mean 
number of customers in the queue is 


P 2 (! + ■£) 

« 2 (1—p) 


(6.4-1) 


and the mean number of customers in the system 


L 


P 2 (l + ^) , 

2 (1—P) + P 


(6.4-2) 
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where, as before, p = X/p, and v ^ is the ratio between 
the standard deviation and the expectation of service 
times. The above expressions are referred to as the 
Pollaczek-Khinchine formulas. Dividing L q and L by 
X, we obtain, by virtue of Little’s formula, the mean 
time a customer waits in the queue and the mean time 
it spends in the system: 


1T 9 = 


P 2 (l + ^) 
2X (1—p) 


(6.4-3) 


W 


P 2 D+^) 1 

2X. (1—p) + p 


(6.4-4) 


Note that in a special case when service times have 
an exponential distribution, = 1 and the formulas 
(6.4-1) and (6.4-2) reduce to the familiar expressions 
(6.3-16) and (6.3-20) for the Poisson single-channel 
queue. 

Take another special case when the service time is 
not random at all so that v ^ = 0. Then the length of 
queue cuts twice as compared with the Poisson case. 
This fact has a physical explanation. If the servicing 
process is more ordered and runs regularly the queueing 
system performs better than with a poorly organized, 
disordered (random) service. 

(iii) Single-server service facility with arbitrary in¬ 
put and arbitrary service times. The single-channel 
system concerned has an infinite population source 
supplying a recurrent input of rate X and coefficient 
of variation of inter-arrival intervals in the range be¬ 
tween zero and unity, i.e. 0<Cu*,<l. The service 
times are also arbitrarily distributed with mean 1/p. and 
coefficient of variation v ^ ranging in a zero-one interval 
too. Whatever explicit analytical formulas for the 
system have not been managed so far. We can, how- 
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ever, estimate the mean length of the queue by bounding 
it from below and from above. This estimation has 
been shown to work with sufficient accuracy 




P 2 (4+^) , Pd-HD 

2(1—P) + 2 

_ p* («*+«$ (1 —p) (1—»i) 

= 2(1 —p) + 2 


(6.4-5) 


When the input process is Poisson, the estimates 
both from above and from below coincide to yield the 
Pollaczek-Khinchine formula (6.4-1). To effect very 
rough estimations, Feinberg [18] derived the simple ex¬ 
pression 


L q 


P* («*+«$) 
2(1—P) 


(6.4-6) 


The average number of customers in the system re¬ 
sults simply if we add the average number of serviced 
customers p to L q : 

A « L q + p (6.4-7) 

The respective mean times in queue and in system 
can he computed by dividing L q and L, in view of 
Little’s formula, by k. 

Thus the performance measures of single-server sy¬ 
stems with unlimited queue size may be found (if 
not exactly, then approximately) also in the cases 
where arrivals are non-Poisson and service times are 
not exponential. 

A natural question may arise: how about the multi¬ 
channel non-Poisson queues? An absolutely frank an¬ 
swer would be: bad. No analytic methods exist for 
these systems. The sole characteristic we may evaluate 
is the average number of busy channels k — p. As far 
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as the other measures, L q , L, W q , and W , are concerned, 
they cannot enjoy any general formula. 

Yet if the channels are actually many (say 5 or 
more), the nonexponential service times will not impede 
the analysis, should only the input process be Poisson. 
To prove, the total process of service completions ren¬ 
dering channels free adds up of the processes of comple¬ 
tions in individual channels; the superposition results, 
as we know, in a process close to the Poisson process. 
Hence in this case the substitution of exponential dis¬ 
tribution for nonexponential service times entails 
comparatively small errors. Fortunately, in many 
applications the input process is close to Poisson. 

The situation is worse when arrivals are far from Poi¬ 
sson. This is a field for a researcher to display his in¬ 
genuity. One possibility is to assign two single-server 
systems of which one is “superior” and the other “in¬ 
ferior” to the queue at hand (the comparison may be 
in terms of queue length and waiting time). For single¬ 
server systems, however, we are able already to eval¬ 
uate the performance measures in any circumstances. 

The “inferior” and “superior” single-server systems 
may be devised in a variety of ways. For example, an 
inferior queue results if a given rc-channel system is 
partitioned into n single-channel queues with the to¬ 
tal arriving input distributed between these systems 
in the order of arrival of the customers, that is, the first 
customer to the first system, the second to the second, 
and so on. We know that each of the systems will then 
receive an n-Erlangian input with coefficient of varia¬ 
tion 1 lY w. The coefficient of variation for service 
times remains the same. We know also how to deduce 
the system time for such a single-server queue, W ; 
it will be automatically longer than that for the orig¬ 
inal n-channel system. With this time on hand we 
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may give a “pessimistic” estimate also for the mean 
number of customers in the queue resorting to Little’s 
formula and multiplying the mean time by the arrival 
rate X of the total input process. 

An “optimistic” estimate results if the re-channel sys¬ 
tem concerned is modeled by one single-channel queue 
with service rate re times quicker than in the original 
system, that is rep. Naturally, parameter p must also 
be diminished re times. The waiting time in the system 
decreases in this queueing system because each service 
is performed re times quicker. Using the adjusted val- 

ue p = p/re, the coefficient of variation for arrivals, 
v x , and the coefficient of variation for service times, 
we can approximately compute the mean number 

of customers in the system L. By subtracting from this 
value the original (nonadjusted) value for p we obtain 

the mean number in the queue L q = L — p. Both 
characteristics will have lower values than they would 
in the original re-channel system (as if estimated on an 
“optimistic” basis). Dividing them by X we can find 
the respective “optimistic” estimates for the system 
time and time in queue. 




Chapter 7 


STATISTICAL MODELING OF RANDOM PROCESSES 
(THE MONTE CARLO METHOD) 


7.1 Idea, Purpose and Scope of the Method 

In the previous chapters we have learnt to set up some 
analytical models of operations under stochastic un¬ 
certainty. These models enable us to establish an ana¬ 
lytical (formal) dependence between the operation con¬ 
ditions, decision variables and the result (outcome) 
of the operation that is characterized by one or more 
performance measures. In many operations, queueing 
(or other similar) systems (e.g., devices with units that 
may fail) appear as “subsystems” or “components” of 
the controlled system. The benefits and desirability of 
developing the analytical models (if only approximate 
ones) are beyond any doubt. Unfortunately, these are 
amenable to construction solely for the simplest, “un¬ 
pretentious” systems, and above all, they call for the 
assumption that the process is Markovian, which is 
not always the case in reality. Where analytical meth¬ 
ods are inapplicable (or their precision is questionable) 
one has to rely on the universal method of statistical 
modeling, or as it is often called the Monte Carlo meth¬ 
od. 

The underlying idea of the technique is extremely 
simple and is as follows. Instead of describing a pro¬ 
cess using an analytical tool (differential or algebraic 
equations), we “play off” a random phenomenon using 
a specially organized procedure involving the random- 
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ness and yielding a random result. In actual fact, the 
random process is realized in each specific case in a 
somewhat different way; the statistical modeling, too, 
gives each time a new, different realization of the pro¬ 
cess examined. What may it give us? As such, next 
to nothing, just as, say, one case of recovery due to 
some medicine (or despite of it). But if these realiza¬ 
tions are many, that’s another pair of shoes. This set 
of realizations may be utilized as some artificially ob¬ 
tained statistical evidence, that lends itself to pro¬ 
cessing with conventional techniques of mathematical 
statistics. This processing may yield (of course, ap¬ 
proximately) any statistic of interest: probabilities, 
expectations, and variances of random variables, and 
so forth. Monte Carlo simulation of random phenome¬ 
na uses the very chance as a tool, makes it to “work for 
us”. 

Not infrequently such a procedure appears simpler 
than attempts to build an analytical model. With 
complicated, clearly non-Markovian operations involv¬ 
ing a ghost of elements (machinery, people, organiza¬ 
tions, auxiliary facilities) with random aspects inex¬ 
tricably entwined, statistical modeling appears, as a 
rule, far simpler than the analytical one (and often 
the only one possible). 

In essence, the Monte Carlo technique can work out 
any stochastic problem, but it is only justified where 
the game procedure is simpler as compared with the 
analytical calculation. We will cite an example of a 
case where the Monte Carlo method is possible, but ut¬ 
terly pointless. Let three independent shots be fired 
at a target so that each shot hits with probability 1/2. 
It is desired to find the probability of at least one hit. 
Straightforward computation gives the probability 
to be 1 — (1/2) 3 = 7/8. The problem, in principle, 
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may also be solved using a chance sampling, or statisti¬ 
cal modeling. We will instead toss “three coins”, tak¬ 
ing heads to mean “hits”, and tails “misses”. A trial 
is considered “successful” if at least one of the coins 
shows head. We will perform very many trials, count 
“successes” and divide the result by the number N of 
trials carried out. We will thus obtain the frequency 
of the event, which at a large number of trials is close 
to probability. What of it? Such a procedure might 
only be used by a person innocent to probability, 
though it is possible in principle 1 . 

And now consider another problem. Let a multichan¬ 
nel queueing system be given such that the process 
in it be clearly non-Markovian: the intervals between 
demands have a non-exponential distribution, the 
servicing times distributed similarly. What is more, 
channels fail now and then and must be repaired; both 
the failing and repair times have non-exponential dis¬ 
tributions. We would like to find the queueing system 
characteristics: state probabilities as time functions, 
the average queue length, the average system time for 
a demand, and so forth. It might appear that the prob¬ 
lem is not so involved. But anyone with a knowledge 
of queueing theory would not hesitate to select for its 
solution the method of statistical modeling (the pros¬ 
pects of developing here a workable analytical model 
are fairly bleak). He would have to simulate a multitude 
of realizations of the random process (to be sure, on 
a computer, not manually) and from such artificial 
“statistics” aprroximately deduce the desired probabil¬ 
ities (as frequencies of appropriate events) and expec¬ 
tations (as arithmetic means of random variables). 

1 Something of the sort happens sometimes when people 
with only a smattering knowledge of probability use the Monte 
Carlo simulation on problems having an analytical solution. 
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The OR problems use the Monte Carlo method in 
three principal roles: 

(i) in the modeling of challenging, complex opera¬ 
tions involving many interacting random factors; 

(ii) in checking the applicability of simpler analyti¬ 
cal methods, and identifying the conditions of their 
use; 

(iii) in working out corrections to analytical rela¬ 
tionships such as “empirical formulas” in engineering. 

In Section 1.3 we have already given a tradeoff of 
analytical and statistical models. We can now go a 
step further and say that a chief disadvantage of ana¬ 
lytical models is that they unfailingly require some as¬ 
sumptions to be made about, say, the process being 
Markovian. The adequacy of these assumptions more 
often than not cannot be estimated without control 
computations, these latter being performed using the 
Monte Carlo method. As a figure of speach, the Monte 
Carlo method in OR problems might be compared to 
a quality control department. On the contrary, sta¬ 
tistical models call for no serious assumptions and ap¬ 
proximations. In principle, “crammed” into a statisti¬ 
cal model can be virtually anything—any distribution 
laws, any degree of system complexity and multiplic¬ 
ity of its states. But the major drawback of the statis¬ 
tical models is their unwieldiness and arduousness. 
A sea of x’ealizations, that are necessary to find the de¬ 
sired parameter to an acceptable accuracy, call for far 
too much computer time. Moreover, the results of sta¬ 
tistical modeling are considerably more difficult to 
grasp than the computations following the analytical 
models, and, accordingly, the solution is far more dif¬ 
ficult to optimize (it has to be groped for blindfold). 
An appropriate combination of analytical and statis¬ 
tical methods in operations research is a matter of 
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skill, intuition, and experience on the part of the re¬ 
searcher. Sometimes analytical methods may be of help 
in describing some “subsystems” within a larger sys¬ 
tem for these submodels to be used as “building blocks” 
to erect then the building of a larger, and more complex 
model. 

7.2 Organizing a Random Sampling Mechanism 

The main constituent element of a statistical model is 
one random realization of the phenomenon beingmodeled, 
e.g., “one case of a machine operating to failure”, “one 
working day of a machine shop”, “one epidemics”, and 
so on. A realization is, as it were, a “copy” of the ran¬ 
dom event with its peculiar chance nature. It is by 
their nature that realizations differ from one another. 

An individual realization is staged using a specially 
developed chance mechanism (algorithm) or chance 
trial. Each time where the course of an event is dictat¬ 
ed by chance, its influence is taken into account not 
by computation, but by a chance trial. 

We elucidate the notion of the chance trial. Let a 
process come to a point when its further development 
(and hence its result) is dependent on whether or not a 
certain event A has occurred. For example, whether a 
shell has hit the target, or whether apparatus is in order, 
or whether an object has been detected, or whether a 
failure has been corrected. Then, by performing a chance 
trial we have to answer the question of whether or 
not the event has occurred. How can you possibly real¬ 
ize the trial? You will have to utilize some mechanism 
of random sampling (e.g., to toss a coin, or cast a die, 
or pull out a badge with a number from a rotating 
drum, or take at random a number from a table). We 
are well familiar with some of the mechanisms of ran- 
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dom sampling (e.g., a tombola). If the trial is carried 
out in order to find out if the event A has occurred it is 
to be so organized that the conditional result of the 
game has the same probability with the event A . How 
this is done will be seen below. Apart from random 
events, the course and outcome of the operation may be 
influenced by a variety of random variables, e.g., the 
time to failure of a device; the service time of a chan¬ 
nel in queueing system; the size of a part; the weight 
of a train arriving to a location; the coordinates of a 
point hit by a shell, and so forth. Using the chance 
trial we can “play” both the value of any random vari¬ 
able and a set of values of several variables. 

We will refer as random sampling to any chance ex¬ 
periment with a random outcome that answers one 
of the following questions: 

1. Has the event A occurred or not? 

2. Which of the events A lt A 2 , ..., A h has occurred? 

3. What value did the random variable X take on? 

4. What population of values did the system of ran¬ 
dom variables A 1 , X 2 , ..., X h take on? 

Any realization of the random event by the Monte 
Carlo method involves a chain of random samplings 
alternating with conventional calculations. These take 
into account the influence of the sampling outcome 
on the further course of the event (in particular, on 
the conditions under which the next sampling will be 
taken). 

A chance trial can be obtained in a number of ways, 
but there is one standard mechanism allowing any form 
of sampling to be realized. Namely, for each form it is 
sufficient to be able to obtain a random number R 
whose values from 0 to 1 are equiprobable 2 . The va- 


4 Or rather have the same probability density. 
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riable R will be called for short the “random number 
from 0 to 1". We will show that using this number we 
are able to stage any of the four kinds of sampling 
mechanism. 

1. Has the event A occurred or not? For this question 
to be answered requires a knowledge of the probability 
p of the event A. We will play the random number R 
from 0 to 1, and if it appears smaller than p, as shown 

P 

_A_ 

.-1' « .J_I_ 

0 R / 

Fig. 7.2-1 

in Fig. 7.2-1, we will consider that the event has oc¬ 
curred, and if more than p, has not. 

And how do we go about it, the reader would ask, 
if the number R is exactly p? The probability of such 
a coincidence may be disregarded. And if it has oc¬ 
curred, go as you please: you may either take any 
“equal” to mean “more” or “less”, or alternatively—this 
will essentially have no effect on the result of the 
modeling. 

2. Which of the events A u A 2 , ..., A * has occurred? 
Let the events A x , A 2 , ..., A h be incompatible and form 
a complete group. Their probabilities p x , p 2 , ..., p h 
will then add up to unity. We break the interval (0, 
1) down into k sections of length p x , p 2 , ..., Ph (Fig. 
7.2-2). The number R falling on a section will imply 
that a respective event has occurred. 

3. What value did the random variable X take on? 
If the random variable X is discrete, i.e. it assumes the 
values aq, x 2 , ..., x h with respective probabilities p x ,- 
p 2 , ... p h , then the case clearly reduces to the preced¬ 
ing one. We will now consider the case where the ran- 




206 Ch. 7 Statistical Modeling of Random Processes 


dom variable is continuous and has a predetermined 
probability density / (a;). For its value to be played it 
is sufficient to perform the following: to pass from the 
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Fig. 7.2-2 

probability density / (x) to the distribution function 
F ( x ) according to the relationship 

X 

F{x)= j f(x)dx, (7.2-1) 

*-oo 

and then to find for the function F its inverse function 
TL We next play the random number R from 0 to 1 
and find its inverse function 

X = W (R) (7.2-2) 

It is trivially shown (we will not do it) that the re¬ 
sultant value of X has the desired distribution / ( x ). 

Figure 7.2-3 shows schematically the play (sampling) 
procedure for the value of X. Played here is the number 
R from 0 to 1, and for it we look for a value of X such 
that F (X) = R (as shown by arrows in Fig. 7.2-3). 

In actual practice we often have to play (sample) 
the value of a random variable having a normal dis¬ 
tribution. For it, just as for any continuous random 
variable, the play rule (7.2-2) remains valid, but we 
may also proceed otherwise (simpler). It is common 
knowledge (from the Central Limit Theorem) that the 
addition of a sufficiently large number of independent 
random variables with the same distribution gives a 
random variable that approximately has a normal dis- 
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tribution. In practice, to obtain a normal distribution 
it suffices to add up six copies of a random number 
from 0 to 1. The sum of these six numbers 

Z = /?! + H 2 -j- ... -j- i?6 (7.2-3) 

has a distribution that is so close to the normal that 
in a majority of practical problems it may be substi- 


F(x) 



tuted for the latter. For the expectation and standard 
deviation of this normal distribution to be equal to 
specified m x , o x , we must subject the variable Z to a 
linear transformation 

X = o x V 2 (Z — 3) + m x . (7.2-4) 

This is exactly the desired normally distributed random 
variable. 

4. What manifold of values did the system of ran¬ 
dom variables Xi, X->, take on? If the random 

variables are independent, it is sufficient to repeat k 
times the procedure just described. With dependent 
variables, each next variable must be played following 
its conditional distribution law, given that all the 
previous ones have assumed the values given by the play 
(we will not dwell upon the case in more detail). 
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We have thus considered all the four forms of sam¬ 
pling and seen that they all come down to the play 
(single or repeated) of the random number R from 0 
to 1. 

A question arises: “But how is this number R 
played?” There exist a number of forms of the so-called 
‘random number generators’ used to tackle the 
problem. Let us dwell a bit on some of them. 

The simplest of the random number generators is 
the rotating drum in which numbered balls (or badges) 
are stirred. Suppose, for example, we want to play 
the random number R from 0 to 1 to within 0.001. We 
charge the drum with 1000 numbered balls, set it 
into rotation and, after it has stopped, select at ran¬ 
dom a ball, read its number and divide it by 1000. 

We might proceed otherwise: instead of 1000 
balls we put into the drum only 10, numbered 0, 1, 
2, ..., 9. We draw a ball, read the first decimal place 
of a fraction. Return it to the drum, reset the drum in¬ 
to rotation, and draw a second ball—this will give us 
the second decimal place, and so forth. It is easily prov¬ 
en (we are not going to do this) that the decimal frac¬ 
tion thus obtained will show the uniform distribu¬ 
tion from 0 to 1. This method lias the advantage that 
it is in no way related to the number of decimal places 
to which we want to know R. 

This invites a suggestion: not to play R each time 
when we need it but to make it beforehand, i.e. com¬ 
pile a fairly extensive table listing in a random way 
all the figures 0, 1, 2, ..., 9 with the same probability 
(frequency). This idea has long been implemented: 
such tables have really been compiled and put into 
practical use. They are called random numbers tables. 
Sections of random numbers tables are given in many 
texts on probability and statistics (e.g., [20]). Also, 
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short passages from these tables are provided in a popu¬ 
lar book by the present author [21] which, by the way, 
cites examples of the modeling of random processes 
using these tables. 

With manual Monte Carlo simulation the random num¬ 
bers tables are the best way of deriving the random 
number R from 0 to 1. If the modeling is computer-as¬ 
sisted, it would be unreasonable to make use of the 
random numbers tables (and tables in general) as these 
would overburden the memory. The computer deri¬ 
vation of R uses appropriate generators built-in into 
most of the computers. These may be both “physical 
generators” which convert random noise, and compu¬ 
tational algorithms utilized by the computer to cal¬ 
culate the so-called “pseudo-random numbers”. The 
prefix “pseudo” here stands for “as it were”. Indeed, 
the numbers produced using these algorithms actually 
are not random but, for all practical purposes, these 
behave as such: all the values from 0 to 1 occur on av¬ 
erage with the same frequency and, besides, there is 
essentially no relation between the successive values 
of the numbers thus obtained. 

There exist a number of algorithms to derive pseudo¬ 
random numbers differing in the degree of simplicity, 
uniformity and other aspects (see [22]). One of the sim¬ 
plest algorithms for computing pseudo-random num¬ 
bers consists in the following. We take two arbitrary 
«-digit binary numbers a x and a 2 , multiply them and 
in the resultant product we take n middle digits—this 
will be the number a 3 . We next multiply a 2 and a a 
and again we take in the product n middle digits, and 
so forth. The numbers thus derived are regarded as a 
sequence of binary fractions with n decimal places. 
Such a sequence of fractions behaves as a series of val¬ 
ues of the random number R from 0 to 1. There are 
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other algorithms as well, which rely not on multipli¬ 
cation but on “summation with a shift”. A more de¬ 
tailed discussion of concrete algorithms for deriving the 
pseudo-random numbers would he pointless: at pre¬ 
sent practically all the computers are either fitted 
with random number generators, or with tested algo¬ 
rithms for computing pseudo-random numbers 3 . 


7.3 Modeling a Stationary Random Process 
by a Single Realization 

In operations research we often have to handle problems 
where the random process lasts rather long and under 
the same conditions and we are just interested in the 
behavior of the process in the steady state. For exam¬ 
ple, a railway sorting yard works 24 hours a day, with 
the intensity of the arriving traffic flow being nearly 
independent of time. Other examples of the systems 
in which a random process fairly quickly comes to a 
stable condition, are computers, communications 
lines, continuously operated apparatus, and so on. 

We have already discussed the limiting stationary 
mode of operation and equilibrium state probabilities 
in Chapter 5 in connection with the Markov random pro¬ 
cesses. Do they exist for non-Markovian processes? 
Yes, they do exist in certain cases and they do not 
depend on the initial conditions. In solving the ques¬ 
tion of their existence for a given problem we can in 


3 This argument suspiciously resembles the reasoning of 
a hero of The Dunce, a comedy by the Russian playwright Denis 
Fonvizin (1745-1792), who thus advocates the uselessness of 
geographical knowledge: “ What the hell are coachmen for? It 
is their business. And the science is not for nobles. A nob¬ 
leman will only have to say: bring me there and there, and they 
will bring him wherever he wishes." 
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a first approximation proceed as follows. We assume all 
the processes being Poisson. If in this case an equilib¬ 
rium distribution exists, it will exist for the non- 
Markovian process, too. If this is the case, all the sta¬ 
tistics for the limiting, stationary conditions may be 
determined by the Monte Carlo method not from a set 
of realizations but from one only, this realization being 
long enough. With this procedure a long realization 
provides exactly the same information about the pro¬ 
cess behavior as the set of realizations with the same 
total duration. ^ 

Let only one long realization of a stationary random 
process of total duration T be available. The state prob¬ 
abilities of interest to us can then be found as a per¬ 
centage time spent by the system in these states, and 
the mean values of the random variable can be ob¬ 
tained by the averaging not over the set of realizations 
but over the time along the realization. 

Consider an example. A single-channel non-Marko- 
vian queue is being modeled by the Monte Carlo meth¬ 
od. The queue has a waiting room only for two de¬ 
mands. A demand arrived when both places in the queue 
are occupied leaves the system unserved (is lost). 
From time to time the channel may fail. In that case, 
the demands in the servicing system (both being served 
and in the queue) do not leave the system but wait for 
the end of the repair. All the processes are non-Poisson 
but arbitrary recurrent. The possible states of the 
queueing system are: 

S oi = the channel is healthy and the system con¬ 
tains i demands, 

Sii — the channel is being repaired, the system con¬ 
tains i demands (i = 0, 1, 2, 3). 

The graph of queueing system states is shown in 
Fig. 7.3-1. The form of the graph suggests that the 


14 * 
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final probabilities do exist. Suppose the queueing sys¬ 
tem operation has been modeled by the Monte Carlo 
technique over a large time span T. We wish to find the 
characteristics of the system performance: P \ 03s is 



Fig. 7.3-1 

the probability of a demand leaving the system un¬ 
served; P h is the probability of the channel being 
healthy; A is the absolute throughput of the system; L 
is the average number of demands in the system; L q 
is the average number of the demands in the queue; 
W and W q are the mean times in the system and 
queue. 

To begin with, we find the equilibrium probabilities 

P oo) Pou P 021 P 031 Pioi Piii Pi 2 i Pi 3 * To this end we cal¬ 
culate along the realization the total time spent by 
the system in each of the states, T 00 , T 01 , T 02 , T 03 , 
Tio> Tin T 12 , T u , and divide each of them by the time 
T. We obtain 

Poi~~p-> Pu — —jr (i — 0, 1, 2, 3) 

The loss probability equals the probability of a de¬ 
mand arriving when there are already three demands 
in the system 


P loss — P 03 4~ Pl3 
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The absolute throughput is 

A = X (1 — P loss ) 

where A is the arrival rate. 

The probability that the channel is healthy is derived 
by summing up all the probabilities whose first sub¬ 
script is zero 

Ph = Poo + Poi + P02 + P03 

The average number of demands in the system is 
calculated by multiplying the possible numbers in the 
system by the respective probabilities and summing up 

L — i (poi + p u) + 2 (p 02 + P12 ) + 3 (pos + P13) 

This amounts to indexing on the time axis the time 
instants (epochs) in which the queueing system con¬ 
tains 0, 1, 2, 3 demands; multiplying the total length 
of the segments by 1, 2, 3 respectively; adding them up 
and dividing by T. 

We obtain 'L q by subtracting from L the average 
number of demands in service (for a single-channel 
queueing system this is the probability of the channel 
being busy) 

^busy = 1 (Poo " 1 " Pio) 

The average lime that a demand spends in the sys¬ 
tem and in the queue will be obtained by Little’s 
formula 



This completes our brief presentation of the Monte 
Carlo method. The interested reader is referred to the 
works [6, 22], which give a more detailed treatment and 
consider, in particular, the accuracy of statistical mod¬ 
eling. 




Chapfer 8 

CAME THEORY FOR DECISION MAKING 


8.1 Subject and Problems of Game Theory 

In the preceding three chapters we have considered the 
issues pertaining to mathematical modeling (and some¬ 
times to solution optimization), in the cases where 
the operation conditions incorporate an uncertainty, 
relatively favorable, however, i.e. stochastic, which 
in principle can be refined knowing the distributions 
(or at least numerical characteristics) of the random 
factors involved. 

Such an uncertainty is only half the trouble. This 
chapter will discuss (necessarily briefly) a far worse 
kind of uncertainty (in Section 2.2 we have dubbed it 
“unfavorable”) where certain parameters contributing 
to the success of an operation are unknown, and there 
is no evidence whatsoever suggesting that these val¬ 
ues are more probable, and those less. Uncertain (in 
the “unfavorable” sense) may be both external, “objec¬ 
tive” conditions, and “subjective” ones, i.e. intention¬ 
al actions of adversaries, rivals or other persons. As 
the saying goes “the human heart is a mistery”, and 
the behavior of those involved may be even more dif¬ 
ficult to predict over a random event domain. 

To be sure, when dealing with an uncertain (unfa¬ 
vorable) situation all the inferences of a scientific study 
can be neither correct nor unique. But in this case, too, 
the quantitative analysis may be of help in arriving 
at a decision. 
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Such problems are the concern of a special branch of 
mathematics with a picturesque name “theory of games 
and decision making”. In certain (rare) cases its 
techniques enable optimal solutions to be found. But 
more often than not these methods allow one just to 
clarify a situation, to evaluate each option from various 
(sometimes conflicting) angles, to analyze its advantages 
and disadvantages and, eventually, to make a deci¬ 
sion, admittedly not uniquely correct, but, at least, 
thoroughly thought over. It is not to be forgotten that 
decision making under uncertainty is bound to involve 
some arbitrariness and risk. Shortage of information 
is always a problem and not an advantage (though it 
is in this situation that the researcher can show off 
his most elegant mathematical techniques). Neverthe¬ 
less, in an involved situation, hard to overview as a 
whole, when the researcher is confronted with an abun¬ 
dance of detail, it is always helpful to represent the 
options in a form such that the selection is less arbi¬ 
trary, and the risk minimal. Not infrequently the prob¬ 
lem is formulated in such a way: What is the price to 
be paid for the information wanted for the operation 
efficiency to be improved? Note that at times the 
decision making does not require accurate informa¬ 
tion as to the conditions, and it suffices to indicate a 
“region” where they lie [23]. 

This chapter gives a brief treatment of the theory of 
games and statistical decisions. For more detailed in¬ 
formation see [24-28]. 

The simplest of the situations containing the “un¬ 
favorable” uncertainty are the so-called conflict situa¬ 
tions. The name refers to the situations with conflict¬ 
ing interests of two (or more) sides pursuing different 
(sometimes opposite) aims, with the gains of one side 
depending on the way in which the other will behave. 
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Examples of conflict situations are legion. These un¬ 
doubtedly include any situation in warfare, a number 
of situations in competitive economies. Conflicting 
interests are involved in courts, sport, and species 
competition. To a certain degree, contradictory are 
interrelationships of various stages in the hierarchy 
of complex systems. Also, conflicting in a sense is a 
situation with several criteria: each places its own re¬ 
quirements to management and, as a rule, these require¬ 
ments are conflicting. 

Game theory is exactly the mathematical theory of 
conflicting situations. «Its objective is to work out 
recommendations for the reasonable behavior of the 
conflict sides. 

Each conflicting situation taken directly from real 
world is extremely involved, its analysis being com¬ 
plicated by a number of factors of marginal importance. 
For a mathematical analysis of a conflict to be pos¬ 
sible, its mathematical model is developed. This mod¬ 
el is called a game. 

Game differs from a real conflict in that it proceeds 
according to some rules. These rules stipulate the rights 
and duties of the participants, as well as the outcome 
of the game—who wins or loses depending on the sit¬ 
uation at the moment. Since time immemorial human¬ 
ity has been using such formalized models of conflicts— 
“games” in the narrow sense of the word (draughts, 
chess, cards, etc.). Hence the name of the theory and 
its terminology: the conflicting sides are called players, 
who “win” or “lose” in the game. We will assume that 
the winnings (losses) of the participants are amenable 
to quantitative expression (if this is not the case, then 
we at all times can assign it to them, e.g., in chess 
take a “victory” to be unity, a “loss” minus one, a 
“draw” zero). 
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In a game interests of two or more participants can 
clash; in the first case the game is called two-person , 
in the second an n-person game. Participants at an 
re-person game can form coalitions (constant or tempo¬ 
ral). One of the tasks of game theory is the identifying 
of reasonable coalitions in the n-person game and de¬ 
veloping of information exchange rules between the 
players; an re-person game with two constant coali¬ 
tions will naturally turn into a two-person game. 

The game course may be represented as a series of 
successive “moves” of the players. A move is a selec¬ 
tion by a player of one of the actions stipulated by the 
rules and putting it into effect. Moves can be personal 
and chance (or random). With a personal move the 
player deliberately chooses and realizes one or another 
alternative (e.g., a move in chess). A chance move is 
made not by the player’s will, but by means of some 
random mechanism (coin tossing, die casting, card 
drawing, etc.). Some games (so-called “games of pure 
chance”) involve random moves only—these are not the 
concern of the theory of games. The objective of the 
latter is the optimization of the player’s behavior in 
the game which (maybe in addition to chance moves) 
includes personal moves as well. Such games are termed 
strategic. 

A strategy is a rule which tells a player what to do, 
i.e. what alternative to choose in each personal move 
depending on the situation at hand. 

A player normally does not follow some strict, “hard 
and fast” rule: the selection (decision) is made in the 
course of the game when directly observing the situa¬ 
tion. Theoretically, however, this makes no difference 
if we suppose that all of these decisions have been made 
beforehand by the player (“in such and such situation I 
will make such and such move”). This will mean that 
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the player has adopted a strategy. Now he may even 
not take part in the game personally, but to give over 
a list of rules to an unbiased person (umpire). The 
strategy can also be fed into an automat in the form of 
a program (it is in this way that computers play chess). 

Depending on the number of strategies games are 
divided into finite and infinite. A game is finite if each 
player has at his disposal only a finite number of stra¬ 
tegies (otherwise the game is infinite). There are games 
(e.g., chess) where the number of strategies is in prin¬ 
ciple finite, but so large that their complete exhaustion 
is practically impossible. 

The optimal strategy is such that provides the best 
situation in the game, i.e. maximal payoff. If the game 
is repeated many times and contains, apart from the 
personal, the random moves as well, the optimal strat¬ 
egy provides the maximal average payoff. 

The task of game theory is to find optimal strategies 
for players. The major underlying idea in finding the 
optimal strategies is that the opponent (in the general 
case, opponents) is at least as adroit as the player him¬ 
self, and does his best to reach his goal. The assumption 
of the intelligent opponent is but one of the possible 
attitudes in a conflict, but the theory of games is based 
just on this assumption. 

A game is said to be a zero-sum game if the total payoff 
to all the players is zero (i.e. each player wins at the 
expense of the others only). The simplest case is the two- 
person zero-sum game. The theory of these games is the 
most advanced section of the theory of games contain¬ 
ing clear-cut recommendations. Below we will 
acquaint ourselves with some of its notions and proce¬ 
dures. 

Game theory just as any mathematical model, is 
not without its constraints. One of them is the assump- 



8.2 Matrix Games 


219 


tion that the opponent (opponents) is perfectly (“ideally”) 
intelligent. In a real conflict the optimal strategy often 
consists in finding out where the opponent is “silly” 
and making use of it to one’s own advantage. Game- 
theoretical schemes do not include risk, that invariably 
accompanies reasonable decisions in real conflicts. 
The theory of games teaches the sides of a conflict to 
be most careful, overcautious. Being aware of these con¬ 
straints and therefore not following blindly the recom¬ 
mendations derived by game theory, we can still make 
a reasonable use of this approach for consultations in 
decision making (just as a young aggressive military 
leader may listen to the opinion of an experienced, 
careful sage). 


8.2 Matrix Games 

The simplest case treated at length in game theory is 
the finite two-person zero-sum game (non-cooperative 
game of two persons or two coalitions). Consider a 
game G with two players A and B showing conflicting 
interests: the gain of the one must be equal to the loss 
of the other. As the payoff to player A is that to playr 
B with the opposite sign, we may concern ourselve 
solely with the gain a of A. It is only natural that A 
is out to maximize, and B to minimize a. For simplic¬ 
ity, we identify ourselves with one of the players (for 
definiteness, A) and will call him “we”, and B, “the 
opponent” (it goes without saying that this gives no 
real odds to A). Let we have m possible strategies 
A t , A 2 , ■ ■ ., A m , and the opponent n possible strate¬ 
gies B t , B 2 , . . ., B„ (such a game is termed an m X n 
game). Denote by a i} our payoff in the case that we 
employ strategy A,, and the opponent, strategy Bj. 
Suppose that we know the payoff (or average payoff) 
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a t ] for each pair of strategies At, Bj. Then, in principle, 
we can set up a rectangular table (matrix) summariz¬ 
ing the players’ strategies with respective payoffs 
(see Table 8.2-1), 


Table 8.2-1 




b 2 

B n 

At 

a ll 

fl l2 

a m 


a 21 

a 22 

a 2 n 

Am 

a mi 

a m 2 

■ ■ a mn 


If such a table is available, game G is said to be re¬ 
duced to matrix form (in itself, the reduction of the 
game to this form maybe a challenging problem, that 
is at times practically intractable due to a spate of 
strategies). It is to be noted that if a game is reduced to 
a matrix form, then a multimove game is actually re¬ 
duced to a single move: the player is only required 
to select a strategy. The game matrix will be denoted 
la t j]. 

By way of example consider a 4 x 5 game in matrix 
form. We can select from four strategies, the opponent 
from five strategies. The game matrix is given in Ta¬ 
ble 8.2-2. 

Let us reflect for a moment on what strategy we 
(player A) are to follow. Matrix 8.2-2 includes a tempt¬ 
ing entry “10”, we feel drawn to select strategy A 3 
whereby this “titbit” will come our way. It is all very 
well, but the opponent is not exactly a fool! Should we 
choose A 3 , he, out of spite, will choose B s that will 
land us a miserable “1”. No, A s will not do! What 
then? Obviously, from the cautiousness principle (cen- 
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Table 8.2-2 



By 

b 2 

b 3 

By 

B b 

Ay 

3 

4 

5 

2 

3 

a 2 

1 

8 

4 

3 

4 

As 

10 

3 

1 

7 

6 

A 4 

4 

5 

3 

4 

8 


tral to game theory) we have to seek a strategy such that 
our minimal payoff be maximal. This is the so-called 
“minimax” principle: make the best of the worst (be¬ 
havior of the opponent). 

We rewrite Table 8,2-2 so that to add on the right 
an additional column, summarizing minimum entries 
of each row, and denote it by a t for the zth row (see 
Table 8.2-3). 

Table 8.2-3 



By 

B, 

B a 

By 

B 5 

«i 

Ay 

3 

4 

5 

2 

3 

2 

a 2 

1 

8 

4 

3 

4 

1 

A a 

10 

3 

1 

7 

6 

1 

a 4 

4 

5 

3 

4 

8 

0 

P/ 

10 

8 

i 

n 

8 



Of all the values of a t (the right column) wc have sep¬ 
arated the largest (3). The corresponding strategy is 
A 4 . Having chosen his strategy, we can in any case be 
sure that (whatever the opponent’s behavior) we will 
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win no less than 3. This is our guaranteed payoff, if 
we play our cards right we will not get less (and maybe 
even more). This payoff is called the minorant value 
of the game (or maximin, the maximum of minimal 
payoffs). We denote it by a. In our case a = 3. 

Now, try to put ourselves into the opponent’s shoes. 
He is not a mere pawn, and has some gumption! 
Choosing a strategy he would like to lose less, but must 
count on our worst (for him) behavior. Should he choose 
B 1 , we would answer with A 3 and he would lose 10; 
if B 2 , we would answer with A 2 , and his loss would be 
8, and so forth. Add to Table 8.2-3 one more lower row 
to list maxima of columns |3 ; -. Clearly, a cautious op¬ 
ponent must choose a strategy minimizing this payoff 
(the corresponding value 5 is separated in Table 8.2-3). 
This fi is the payoff that a clever opponent will clearly 
not allow us. It is called the majorant value of the game 
(or minimax, the minimum of maximal payoffs). In 
our example p — 5 and the corresponding opponent’s 
strategy is B 3 . 

Thus, according to the cautiousness principle (the 
rule: always count on the worst!), we have to select 
strategy A 4 , and the opponent B 3 . Such strategies are 
called minimax strategies (proceeding from the mini¬ 
max principle). As long as both sides in our example 
follow their minimax strategies, the payoff will be 
a i3 = 3. 

Now suppose for a moment that we found out that 
the opponent follows B 3 , Let us punish him for it and 
choose A x and win 5 —not bad. But the opponent, too, 
has got his wits about him. He found that our strat¬ 
egy is and chosen B x , thus reducing our payoff to 
2, and so forth (the sides began to try strategies like 
crasy). In a word, minimax strategies in our example 
are unstable in relation to information about the behav- 
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ior of the opposite side—these strategies exhibit no 
equilibrium. 

Is it always the case? No, not always. Look at the 
example with the matrix tabulated in Table 8.2-4. 

Table 8.2-4 



Bi 


B s 

B\ 

Cti 

^1 

2 

4 

7 

5 

2 


7 

6 

! 8 

7 

6 


^3 

5 

3 

4 

1 

1 

Pi 

7 ! 

6 

8 

7 



In this example the minorant game value is equal to 
the majorant one: a = |3 = 6. What is the import of 
this? Minimax strategies of A and B will be stable. As 
long as both of them follow these strategies, the payoff 
is 6. Let us see what will occur if we (A) find out that 
the opponent (B) abides by j 3 2 ? Nothing whatsoever 
will change. Just because any deviation from strategy 
A 2 may only impair our situation. In exactly the same 
way, information obtained by the opponent will not 
make him abandon his strategy B 2 . The pair of A 2 
and B 2 has the property of equilibrium (balanced pair), 
and the payoff (in this case 6) for this pair is called the 
saddle point of matrix 1 . An indication of a saddle point 


1 The term ‘saddle point’ has been borrowed from geomet¬ 
ry where it means a point of the surface at which simultaneously 
a minimum is achieved in one coordinate, and a maximum in 
the other. 











224 


Ch. 8 Game Theory for Decision Making 


and balanced pair of strategies is the equality of the 
minorant and majorant game values with the total 
value of a and fi being called the value of the game. 
We will denote it by V 

a - p = V (8.2-1) 

The strategies ABj (here A 2 , B 2 ) providing this 
payoff are termed the optimal pure strategies, and their 
set called the solution of the game. As to the game itself, 
it is said to be solved in this case in pure strategies. The 
both sides A and B may be supplied with optimum 
strategies for which their position is the best of the pos¬ 
sible ones. If player A in the process wins 6, and player 
B loses 6—what of it, such are the conditions of the 
game: favorable for A and unfavorable for B. 

The reader may ask a question: why then the opti¬ 
mal strategies are called “pure”? Forestalling events a 
bit, we answer this question: there are “mixed” strate¬ 
gies, too, in which the player utilizes not one strategy, 
but several strategies, combining them in a random way. 
Thus, if we assume that, apart from pure strategies, 
there are also mixed ones, then each finite game may 
have a solution, the equilibrium point. But this will 
be the subject of our discussion below. The presence of 
a saddle point in a game is not a rule, rather an excep¬ 
tion. A majority of games have no saddle point. How¬ 
ever, there is a class of games that always have a saddle 
point and, accordingly, are solved in pure strategies. 
These are the so-called perfect-information games, i.e. 
such games in which each player in each personal move 
knows all the prehistory of its development, i.e. 
the results of all the preceding moves, both personal 
and chance, Examples of perfect-information games are 
draughts, chess, and so forth. 

It is proved in game theory that each perfect-in- 
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formation game has a saddle point, and hence is solved 
in pure strategies. In each perfect-information game 
there exists a pair of optimal strategies leading to a 
stable gain equal to the game value V. If such a game 
consists of personal moves only, then in using by each 
player of his optimal strategy, it is bound to end in 
quite a definite way—by a payoff equal to the game 
value. Hence, if the game solution is known the very 
game becomes pointless! 

Take an elementary example of a perfect-information 
game. Two players place alternatively identical coins 
on a round table selecting arbitrarily the position of 
the coin centre (the overlapping is not allowed). The 
winner is the one who will place the last coin (when 
there will be no room for other coins). It is trivial to see 
that the outcome of the game is, in essence, predestined. 
There is a definite strategy whereby wins the player 
who is the first to place his coin. Namely, he must at 
first place his coin at the table centre and then the op¬ 
ponent answers to each move by a symmetrical one. It 
is obvious that whatever the behavior of the opponent, 
he cannot escape the loss. The situation with chess and 
perfect-information games in general is exactly the 
same: each of them, written in matrix form, has a saddle 
point, and hence a solution in pure strategies, thus 
it only makes sense as long as the solution is not found. 
For instance, a game of chess either at all times ends in 
a victory of White, or at all times in a victory of Black, 
or at all times in a draw, but which of these—we do 
not know as yet (fortunately for chess lovers). Moreover, 
it is unlikely that we will know it in the foreseeable 
future, as the number of strategies is ever so enormous 
that it is exceedingly difficult to reduce the game 
to a matrix form and find in it a saddle point. 


15-0432 
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This invites the question: What is to be done if the 
game has no saddle point (a |3)? Well then, if each 
player is committed to choose a single pure strategy, 
there is nothing to be done about it: one is to be guided 
by the minimax principle. The situation changes if 
we are able to mix our strategies, choosing them at ran¬ 
dom with some probabilities. The use of mixed strate¬ 
gies is thought in such a way: the game repeats many 
times; before each set, when the player can make a 
personal move, he relegates the choice to chance, by 
a kind of trial choosing tho strategy that has resulted 
(how to organize the random mechanism we know from 
the preceding chapter). 

In game theory mixed strategies constitute a model 
of variable, flexible tactics where neither of the players 
knows how the opponent will behave in this set. Such 
tactics (true, normally without any mathematical reason¬ 
ing) is widely used in card games. It is to be noted 
here that the best way of concealing from the opponent 
your behavior is to make it seem random so that he 
cannot guess beforehand how you are going to move. 

Thus, let’s speak about mixed strategies. We will 
denote the mixed strategies of A and B respectively by 
sA. ~ (Pli Pi* • • •* Pm)i & B ~ (Qli ? 2 i • • •» Qn)i 

where p x , p 2 , . . . p m (which add up to unity) are the 
probabilities of player A using the strategies A x , A 2 , ... 

. . ., A m , and q 1} q 2 , . . ., q n are the probabilities of 
player B using the strategies B 1 , B 2 . . B n . In the 
special case where all the probabilities, safe for one, 
are zero, and this one is unity, the mixed strategy goes 
over into a pure strategy. 

The basic theorem of game theory states: any finite 
two-person zero-sum game has at least one solution —a 
pair of optimal strategies, generally mixed (S%, <Sb>) 
and an appropriate game value V. 
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The pair of optimal strategies (Sa, S%) constitut¬ 
ing the game solution has the following property: if 
one of the players follows his optimal strategy, then the 
other cannot benefit by abandoning his strategy. This 
pair of strategies forms in the game a certain equilib¬ 
rium: one player wants to maximize his payoff, the 
other to minimize, so that each is pulling his way with 
the result that, if they play their cards right, an equi¬ 
librium sets in with a stable gain V. If V > 0, then 
the game is favorable for us, if V < 0, for the opponent; 
at V — 0 the game is fair, i.e. equally favorable for 
both participants. 

We now consider the example of a game without 
a saddle point and give (without proof) its solution. 
The game consists in the following: two players A and 
B simultaneously and arbitrarily show one, two, or 
three fingers. The outcome is decided by the total 
number of fingers shown: if it is even, A wins and 
receives from B a sum equal to the number; if odd, 
the other way round, i.e. A pays to B the sum equal 
to the number. What is to be recommended to the 
players? 

Construct the game matrix. In one game each player 
has three strategies: to show one, two, or three fin¬ 
gers. The 3x3 matrix is given in Table 8.2-5, where 
the additional right column lists the minima of rows, 
and the additional lower row, the maxima of columns. 

The minorant game value is a = — 3, which cor¬ 
responds to strategy A x . This means that with rea¬ 
sonable, careful behavior we may rest assured that we 
lose not more than 3. That’s not much consolation, 
but still better than the payoff 5 in some matrix ele¬ 
ments. Bad luck for us, i.e. player A... But the oppo¬ 
nent’s situation seems to be yet worse: the majorant 
game value is p — 4, i.e. he will lose to us a minimum 


15 * 
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Table 8.2-5 



2?i B 2 B 3 

a; 

dl 

Ao 

ds 

2—3 4 

—3 4 —5 

4 —5 6 

1 

CO 

tO to 

1 ! 

P> 

j 4 j 4 6 



of 4 provided he behaves reasonably. The situation 
in general is not very bright—for either side. Let’s 
see then if we could improve it. It appears that we 
could. If each side follows not some single pure strat¬ 
egy, hut a mixed one, in which the first and third ones 
enter with probabilities 1/4, and the second with 
probability 1/2, i.e. 

S* A = (1/4, 1/2, 1/4), S% = (1/4, 1/2, 1/4) 

then the mean gain will be stably zero (hence the game 
is fair and equally favorable for either side). Strategies 
Sa, S% form the solution to the game, and its value 
is V — 0. How have we arrived at that? That is anoth¬ 
er question. We will show in the following section how 
finite games are solved. 


8.3 Resolving Finite Games 

Before setting out to solve an m X n game we should 
first of all try and simplify it by getting rid of extra¬ 
neous strategies, in much the same way as we did elim¬ 
inating obviously inferior alternatives in Section 
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2.3. We now introduce the notion of domination. 
Strategy A* of player A is said to be dominating 
over Ah, if row A t contains payoffs no less than appro¬ 
priate entries of row A h , and of these at least one is 
actually larger than the appropriate entry of A h . 
If all the payoffs of Ai are equal to corresponding en¬ 
tries in Aft, then strategy A, is said to be duplicating 
Aft. For strategies of player B, domination and du¬ 
plication are defined similarly: dominated is the strat¬ 
egy in which all the payoffs are not larger than appro¬ 
priate entries of the other strategy, and at least one 
of them is actually smaller; duplication implies the 
complete similarity between columns. Naturally, if 
for a strategy there is a dominating one, then the form¬ 
er strategy may be discarded, as may be duplicating 
strategies. 

We clarify the above with an example. Let a 5 x 5 
game be defined by the payoff matrix in Table 8.3-1. 

To begin with, we note that strategy A 5 duplicates 
A 2 therefore any of them may be discarded. We notice, 
on discarding A 5 , that row A x has all the entries larger 
than, or equal to, appropriate payoffs in A 4 , with the 
result that A 4 will dominate over A 4 . Discarding A 4 
gives the 3x5 matrix of Table 8.3-2. 


Table 8.3-1 



ih 

B 2 

b 3 

B* 

B s 

Ax 

4 

7 

2 

3 

4 

A t 

3 

5 

6 

8 

9 

A s 

4 

4 

2 

2 

8 

A 4 

3 

6 

1 

2 

4 

A& 

3 

5 

6 

8 

9 
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Table 8.3-2 



Bx 

b 2 

B a 

Bx 

B s 

A x 

4 

7 

2 

3 

4 

A, 

3 

5 

6 

8 

9 

A, 

4 

4 

2 

2 

8 


Table 8.3-3 



Bx B, 

•^1 

4 2 

a 2 

3 6 

A 3 

4 2 


Table 8.3-4 



Bx 

b 3 

Ax 

4 

2 

a 2 

3 

6 


But this is not all there is to it! A quick glance at 
Table 8.3-2 shows lhat certain strategies of B dominate 
over the others: for example, B 3 over 5 4 and S 5 ; and 
Bi over Z? 2 (do not forget that B seeks to lose the least!). 
Discarding columns B 2 , S 4 , and 5 5 gives the 3x2 
game of Table 8.3-3. 

Lastly, in Table 8.3-3 row A 3 duplicates A x , there¬ 
fore it can be discarded. We eventually arrive at the 
2x2 game of Table 8.3-4. 

Try as you may, this game is not simplifiable any 
further. We will have to solve it. Note in passing 
that rejecting dominated (duplicating and clearly 
unfavorable) strategies in a game with a saddle point 
gives a solution in pure strategies. But it is a good 
idea to test at the beginning whether or not the game 
has a saddle point—this is simpler than to compare 
termwise all the rows and columns. 
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Texts on game theory general! y concentrate on solv¬ 
ing the simple 2 x 2, 2 x re, and m x 2 games that 
permit of a geometric interpretation. But we are not 
going to do all that—instead we will “take the bull 
by the horns” and show the way in which any to x re 
game is solved. 

Let there be a to X re game without a saddle point 
as given by matrix [a u ] of Table 8.3-5. 


Table 8.3-5 



P\ 



Bn 

At 

«n 

°12 

• • • 

°in 

A 2 

a 2l 

a 22 

• • • 

a 2 n 

• • • 

• • • 

• • • 

• • • 

• • • 

Am 

a m l 

a m 2 

• * • 

a mn 


We assume that all the payoffs a i} are positive (this 
can always be achieved by adding to all the matrix 
elements a sufficiently large number M; this will in¬ 
crease the value of the game by M leaving the solu¬ 
tion Sa , S% unchanged). If all a tj are positive, then, 
of course, the game value, i.e. the mean payoff with 
an optimal strategy, will be positive, too. 

We wish to find the solution of the game, i.e. two 
optimal mixed strategies 

Sa ~ (/h> P2> • • •> Pm)i Sg = (<7n • • •> Qn) 

( 8 . 3 - 1 ) 

which give to each side the maximally possible mean 
payoff (minimal loss). 

We first find Sa- We know that if one of the players 
(here A) uses its optimal strategy, then the other (B) 
can in no way improve his situation by abandoning 
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his optimal strategy. Let us force the opponent (B) 
to forsake his optimal strategy by using the pure strat¬ 
egies Bi, B 2 , . . ., B n (while we tenaciously stick 
to S%)- In any event, our payoff will be no less than V 

a ]lP l 4 a ilPi 4 • • • 4 a mlPm ^ V 
a uPl “1” a 2iPl "I - • • • 4 a m2Pm ^ V 

(8.3-2) 

^lnPl 4 Q-mPi “l" • • • "4" ®mnPm ^ ^ 

We now divide the inequalities (8.3-2) by a positive 
quantity V and introduce the notation 

x, = p,/F, x 2 = p 2 /V,..., x m —p m /V (8.3-3) 
The conditions (8.3-2) will then take the form 

^ 11*1 4 ^ 21*2 4 • • • 4 a ml X m ^ I 
® 12*1 4 ® 22*2 4 • • • 4 I 

(8.3-4) 


a ln*l 4 ®2n*2 4 • • • 4 ^ I 

where x x , :r 2 , . . x m are nonnegative variables. By 
virtue of (8.3-3) and the fact that p 1 + p 2 + • • • 
+ p m = 1, the variables x x , x 2 , . . ., x m meet the 
condition 

*1,4- *2 + • • • + x m = 1/V' (8.3-5) 

Yet F is nothing but our guaranteed payoff. It is 
only natural that we seek to make_ it maximal, and 
hence the quantity 1/F is minimal. The solution has 
thus reduced to a mathematical problem: find non¬ 
negative values of the variables x u x 2 , . . ., x m such 
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that minimize the linear function of these variables 
L = x t x 2 + ... + x m min 

(8.3-6) 

subject to the linear constraints (8.3-4). The reader will 
say, “Bah, something familiar!” Exactly—this is 
nothing but a problem of linear programming. The 
problem of solving the m x n game has thus reduced 
to a linear program with n constraint inequalities 
and m variables. Knowing x u x 2 , . . ., x m , we can, 
by (9.3-3), find p x , p 2 , . . ., p m , and hence the opti¬ 
mal strategy Sa and the value V. 

The optimal strategy of player B is found in exactly 
the same way, with the only difference that B seeks 
to minimize, not maximize, the payoff, and, accord¬ 
ingly, to maximize (not minimize) the quantity 1/F, 
and the inequalities instead of ^ will be of ^ type. 
The pair of problems of linear programming seeking 
the optimal strategies ( S% , S%) is called the mutually 
dual problems of linear programming (it has been prov¬ 
en that a maximum of a linear function in one of them 
is equal to a minimum of a linear function in the other 
so that everything is in order—we still do not arrive 
at different values of the game value). 

Thus, solving a m. X n game is equivalent to solving 
a problem of linear programming. Tt is to be noted that 
the opposite holds: for any problem of linear programm¬ 
ing an equivalent game-theoretical problem can be 
formulated (we are not going to discuss the procedure 
here). This interrelation of game problems with those 
in linear programming appears to be of help not only 
for the theory of games, but for linear programming 
as well. The point is that there exist approximate nu¬ 
merical methods of solving games that in certain cases 
(for the problem of a larger size) turn out to be 
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simpler than “classical” methods of linear program¬ 
ming. 

We will next describe one of the simplest numerical 
methods of game solving, the so-called iteration meth¬ 
od. Its underlying idea is as follows. A “thought ex¬ 
periment” is played, in which the sides A and B use 
in turn their respective strategies seeking to win as 
much as possible (lose as little as possible). The ex¬ 
periment consists of a number of plays. It begins with 
one of the players (say, A) choosing at random one of 
its strategies A t . The opponent (B) responds with a 
strategy that is the most unfavorable for Bj, i.e. 
minimizes the payoff to Aj. Now it is again the turn 
of A: he answers B following the same strategy A*, 
that maximizes the payoff for the opponent’s strategy 
Bj. Further, the move of the opponent comes. He 
uses his strategy that is the worst not for our last strat¬ 
egy Ahi but for a mixed strategy that includes strate¬ 
gies Ai, Aft, used so far, with equal probabilities. 
Thus at each step of the iteration process each player 
responds to a given move of the opponent by the 
strategy 

that is optimal for him with respect to a mixed 
strategy of the other which includes all the strat¬ 
egies used thus far in proportion to the frequen¬ 
cies of their application. 

Instead of computing the mean gain each time he may 
simply use the gain “accumulated” throughout the 
preceding moves and select his strategy so that this 
accumulated payoff be maximal (minimal). It has 
been proven that such a technique “converges”: in 
increasing the number of plays the mean payoff per 
play will tend to the value of the game, and the 
frequencies of using the strategies to their respective 
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probabilities in optimal mixed strategies of the 
players. 

But the best way of grasping the iteration method is 
by using a specific example. We will illustrate it re¬ 
ferring to the 3x3 game in the preceding section 
(Table 8.2-5). In order not to deal with negative num¬ 
bers we add 5 to the matrix elements in Table 8.2-5 
(see Table 8.3-6), thus increasing the value of the game 
by 5 with the solution Sa, Sb remaining unchanged. 


Table 8.3-6 



Bi 

B 

Bs 

A t 

7 

2 

9 

A 2 

2 

9 

0 

As 

9 

0 

11 


Let’s begin with some arbitrarily selected strategy 
of player A, e.g., A 3 . Table 8.3-7 lists the first 15 
steps of the iteration process (the reader can carry on 
the computation on his own). The first column con¬ 
tains the numbers of plays (pairs for selection) k; the 
second, number i of the strategy selected in this game 
by the player A; the other three columns, the accumu¬ 
lated payoff in the first k plays for the strategies used 
by the player in the preceding plays and strategies 
Bi, B 2 , B 3 of player B in a given play (derived by add¬ 
ing the elements from an appropriate row to the en¬ 
tries of preceding row). Of these accumulated gains in 
Table 8.3-7 the minimal one is underscored (if they 
are several, all are underscored). The number under¬ 
scored determines the choice by B in this play—he 
chooses the strategy that corresponds to the number 




236 


Ch. 8 Game Theory for Decision Making 


underscored (if there are several such numbers, any 
one of these is taken). We thus determine the number 
/ of the optimal (in this play) strategy B (placed into 
the following column). The subsequent three columns 
list the payoffs accumulated in k games respectively 
at strategies A x , A 2 , A 3 of player A (obtained by add¬ 
ing the elements of column Bj to what was in the row 
above). Of these values in Table 8.3-7 the maximal 
have an overhead bar. It is this value that determines 
the choice by A of the strategy in the following play 
(the line below). The last three columns of Table 8.3-7 

Table 8.3-7 


h 

i 

Bi 

»2 

B 3 

j 

M 

A a 

M 

V 

V 

V* 

t 

3 

9 

0 

11 

2 

2 

9 

0 

0 

9 

4.5 

2 

2 

11 

9 

11 

2 

4 

18 

0 

4.5 

9 

6.75 

3 

2 

13 

18 

11 

3 

13 

18 

11 

3.67 

6 

4.84 

4 

2 

15 

27 

11 

3 

22 

18 

22 

2.75 

5.50 

4.13 

5 

1 

22 

29 

20 

3 

31 

18 

33 

4.00 

6.60 

5.30 

6 

3 

31 

29 

31 

2 

33 

27 

33 

4.84 

5.50 

5.17 

7 

1 

38 

31 

40 

2 

35 

36 

33 

4.43 

5.14 

4.79 

8 

2 

40 

40 

40 

2 

37 

45 

33 

5.00 

5.61 

5.30 

9 

2 

42 

49 

40 

3 

46 

45 

44 

4.45 

5.11 

4.78 

10 

1 

49 

51 

49 

1 

53 

47 

53 

4.90 

5.30 

5.10 

11 

3 

58 

51 

60 

2 

55 

56 

53 

4.64 

5.09 

4.87 

12 

2 

60 

60 

60 

2 

57 

65 

53 

5.00 

5.41 

5.20 

13 

2 

62 

69 

60 

3 

66 

65 

64 

4.61 

5.07 

4.84 

14 

1 

69 

71 

69 

1 

73 

67 

73 

4.93 

5.21 

5.07 

15 

3 

78 

71 

80 

2 

75 

76 

73 

4.74 

5.06 

4.90 
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provide: the lower estimate of the game value V that 
is equal to the minimal accumulated gain divided by 
the number of plays k; the upper estimate of the game 
value V that is equal to the maximal accumulated 
gain, divided by k; and the arithmetic mean of the 
both (it constitutes a better estimate of the value of 
the game than either the minorant or the majorant 
estimates). 

The quantity V * is seen to be varying slightly about 
value of the game V = 5 (the value of the initial game 
was 0, but we added 5 to each matrix element). Using 

Table 8.3-7 we calculate the frequencies p x , p 2 , p 3 , 
Qii Qii 9a °f strategies for the gamblers. We thus obtain 

Pi - 4/15 « 0.26G, p 2 - 7/15 » 0.4G8, 

p 3 = 4/15 » 0.266 q x = 2/15 « 0.133, 

q 2 = 8/15 ^ 0.534, q a = 5/15 « 0.333 

which is not so much different from the values of prob¬ 
abilities p,, p 2 , p o, o., q.y, q 3 that, as indicated above, 
are 1/4 = 0.25, 1/2 - 0.50, 1/4 = 0.25 for the first, 
second, and third strategies, respectively. Such re¬ 
latively good approximations have been obtained at 
as few as 15 iterations—not bad! Unfortunately, the 
approximation process proceeds further not at this 
pace. For the iteration method the convergence, as ex¬ 
perience shows, is very slow. There are ways of spurr¬ 
ing the dragging process, that lie beyond the scope of 
the book. 

The iteration method for game solving has the very 
important advantage that its labour-consumption 
grows relatively slowly with increasing game size 
(m x n) whereas that of the linear programming 
method grows with size far more rapidly. 
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The reader has thus got some insight into the theory 
of noncooperative games and methods of solving ma¬ 
trix games. 

A few words of caution about this theory and its 
applicability are in order here. At the time of its con¬ 
ception the theory of games was looked to for the 
selection of decisions in conflicting situations. The 
hopes were justified but to a slight degree. 

Above all, in actual practice strictly non-cooperative 
games are a not very frequent occurrence—may be 
only in real games (draughts, chess, cards, etc.). Save 
for these artificial situations where one side seeks at 
all costs to maximize, and the other to minimize 
the payoff, such conflicts hardly can occur. It would 
seem that warfare is a field ideally suited for game 
theory. After all, here we encounter the most “fierce” 
antagonisms and sharply conflicting interests 1 But 
the conflict situations in this field appear to be but 
rarely reducible to two-person zero-sum games. Non- 
cooperative scheme is, as a rule, only applicable to 
small-scale operations of marginal importance. For 
example, side A is a group of aircraft attacking an 
object, side B is the antiaircraft defence of the object, 
with the first seeking to maximize the probability of 
destroying the target, and the second to minimize. 
Here the scheme of the two-person zero-sum game can 
find its application. Now consider a more complex 
example: two forces of units (types of tanks, aircraft, 
ships) are engaged. Each side wishes to hit as many 
hostile combat units as possible. In that case the sit¬ 
uation loses the purity of antagonism: it is no longer 
reducible to a two-person zero-sum game. If the sides’ 
goals are not directly conflicting but just not coin¬ 
cident, the mathematical model becomes more involved: 
here we are not satisfied just with a victory, which 
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gives rise to a so-called “bimatrix game”, where each 
side seeks to maximize its gain and not just to mini¬ 
mize the opponent’s gain. The theory of such games 
is very much more sophisticated than that of non- 
cooperative games and, above all, this theory yields 
no distinct recommendations as to the optimal beha¬ 
vior of the sides [26]. 

The second criticism refers to the notion of ‘mixed 
strategies’. With recurrent situations, in which each 
side can easily (without additional expenditures) 
vary its behavior from case to case, optimal mixed 
strategies can really improve the gain in the mean. 
But situations may be in which it is necessary to take 
only one decision (e.g., to choose a scheme of a defen¬ 
sive system). Would it be reasonable to “relegate the 
choice to chance”—roughly speaking, to toss a coin 
and, say, for heads select one form of the design, and 
for tails another? There is hardly a manager who in a 
complicated and crucial situation will dare to make 
his choice in a random manner, even if following the 
principles of games theory. 

Lastly, one more point: it is considered in game 
theory that each player knows all the possible strat¬ 
egies of the opponent and only does not know which 
of them will be used in a given game. In a real world 
situation this is generally not the case: it is just the 
list of the possible strategies of the opponent that is 
unknown, and the best solution in a conflict situation 
will often be just to go beyond the strategies known to 
the opponent, to “bewilder” him by something new, 
unpredictable! 

It is seen from the above that game theory has many 
weaknesses to serve as a basis for decision making 
(even in a highly conflicting situation). Is this to 
suggest that it is not worth studying and is of abso- 
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lulely no use in operations research? No, not at all. 
Above all, the theory is valuable in the very formula¬ 
tion of problems that teaches, in selecting an alterna¬ 
tive in a conflict situation, not to forget that the oppo¬ 
nent, too, is intelligent and to take into account his 
possible tricks and stratagems. Admittedly, recommen¬ 
dations derived from a game treatment are not always 
delinite and not always realizable—it is still useful, 
in selecting a decision, to be guided, among other 
things, by a game model. One only should not think 
of the conclusions drawn from this model as definitive 
and unquestionable 2 . 

8.4 Statistical Decision Analysis 

A close relative of game theory is tire theory of statis¬ 
tical decision-making. It differs from game theory in 
that an uncertain situation has no conflict tinge— 
no confrontation. In problems of decision theory the 
unknown conditions of the operation depend not on 
the deliberately acting “adversary” (or other sides of 
the conflict), but on objective reality that in decision 
theory is generally referred to as “nature”. Pertinent 
situations are often called “games with nature”. “Na¬ 
ture” is thought of as a certain unbiased entity (in 
the words of A. S. Pushkin “indifferent nature”), whose 
“behavior” is unknown, but in any case is not malevo¬ 
lent. 

It might appear that the absence of deliberate coun¬ 
teraction would simplify the choice of alternative. 
But it turns out that it is not so: it complicates, not 


2 The author’s opinion on the role and significance of game 
theory is by no means generally accepted. On the contrary, some 
authors view game treatments in OR as basic ones (see, e.g., [28]). 
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simplifies. Granted the life of the decision-maker in 
a “game with nature” is easier indeed (nobody inter¬ 
feres!), but it is more difficult for him to substantiate 
his choice. When dealing with a conscious opponent, 
the element of uncertainty is partly removed by the 
fact that we “think” for the opponent, “make” decisions 
for him that are the most unfavorable for us. But in 
a game with nature such a concept will not do: God 
knows how nature is going to behave. Therefore de¬ 
cision theory is the most “shaky” science as far as 
recommendations are concerned. Still it has the right 
to existence and is worth attention on the side of OR 
people. 

Consider a game with nature: we (side A) have m 
possible strategies A u A 2 , . . ., A m ; as to the state 
of nature, we may make n assumptions N x , N 2 , . ■ ■ 
. . ., N n . Let’s think of them as “strategies of nature”. 
Our gain a t j for each pair of strategies A t , Nj is given 
by a matrix (Table 8.4-1). 


Table 8.4-1 




A r 2 

N n 


°n 

<z 12 

a m 


a 22 

a 2 2 

a 2 n 

■Am 

a m i 

a m2 

a mn 


It is required to choose a strategy of player A (pure 
or maybe mixed, if possible) that will be more fa¬ 
vorable as compared with the others. 

On the face of it, the problem is similar to a game 
,of two players A and N with opposite interests and 
is solvable by the same techniques. But this is not 


16-0432 
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quite so. The absence of the opposition on the side 
of nature makes the situation qualitatively 
different 3 . 

Let’s give the problem some thought. The simplest 
case of choice in a game with nature is the case where 
(happily!) one of the strategies of A dominates the 
others, as, for instance, A 2 in Table 8.4-2. Here the 


Table 8.4-2 



N i 


N s 

N 4 

Aj, 

1 

2 

3 

5 

A* 

7 

4 

4 

5 

A s 

3 

4 

4 

1 

a 4 

7 

4 

2 

2 


gain with A 2 for any state of nature is not less than 
with other strategies, and for some of them even more. 
It follows thus that it is precisely this strategy that 
must be selected. 

If even the matrix of a game with nature has no 
dominance, it is worthwhile to see if it has duplicating 
strategies or strategies inferior to others under all 
conditions (as we did earlier reducing the game matrix). 
But here there is one subtlety: in such a way we can 
only reduce the number of strategies of player A, but 
not of N —he is absolutely indifferent as to how much 
we gain! Suppose that we have reduced the matrix 


3 Unfortunately, instances are not rare when those with only 
a smattering knowledge of operations research, when meeting 
in practice such a situation, forget about the “indifference” of 
nature and begin immediately to apply techniques of non- 
cooperative game theory. Such recommendations occur in texts 
(mostly popular science). 
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and now it is free of duplicating and clearly unfavo¬ 
rable strategies. 

But what is to be used as a guidance in decision¬ 
making? Naturally, the payoff matrix [a ;j ] must be 
taken into account. However, the picture given by 
matrix [a t j] is in a sense incomplete and does not ade¬ 
quately reflect advantages and disadvantages of each 
decision. 

Clarify this idea that is not simple at all. Suppose 
that payoff a i; - for our strategy A t and state of nature 
Nj is higher than for A h and N t : a i; £> a hi . But why 
so? Maybe we were successful in selecting the strategy 
A £ ? Not necessarily. Or, maybe the state of nature 
Nj is simply more favorable for us than 7V ; . For exam¬ 
ple, the state of nature realized in “normal conditions” 
is for any operation more favorable than, say, “flood”, 
“earthquake”, and so forth. It would be of help to in¬ 
troduce indices such that would not just give the payoff 
for a given situation and strategy, but also show wheth¬ 
er the given strategy in the given situation is chosen 
“happily” or “unhappily”. 

To this end, in decision-making a notion of risk 
is introduced. The risk premium r t j of player A in using 
strategy A t under conditions Nj is the difference be¬ 
tween the payoff that would be obtained if we knew 
the conditions Nj and the payoff that we will obtain 
not knowing them and selecting A t . 

It is obvious that if we (player A) knew the state of 
nature Nj, we would select a strategy maximizing our 
payoff. This payoff is maximal in column Nj. We have 
earlier discussed it and designated as p ; -. The risk 
premium r t j is derived by deducing from P^ the actual 
payoff a tj 


r ij = P; — a t j. (8.4-1) 

16 * 
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By way of example, consider a payoff matrix [a^] 
(Table 8.4-3) and construct for it the risk premium 
matrix [r^] (Table 8.4-4). 


Table 8.4-3 



Nl 

Ns 

Ns 

N, 

A t 

1 

4 

5 

9 

As 

3 

8 

4 

3 

As 

4 

6 

6 

2 

Pj 

4 

8 

6 

9 





Table 8.4-4 



Ns 

Ns 

N 4 

a i 

3 

4 

1 

0 

As 

1 

0 

2 

6 

As 

0 

2 

0 

7 


The examination of Table 8.4-4 brings out certain 
aspects of a given “game with nature”. So, the payoff 
matrix (Table 8.4-3) has in the second row the 
first and last elements equal: a 21 = a 24 = 3. These 
payoffs, however, are far from being equivalent for 
the successful selection of strategies: for the state of 
nature we could gain at most 4, and our choice 
A 2 is almost ideal; but for N 4 we could, by selecting 
A x , get more by as much as 6, i.e. the choice vl a is 
very bad. Risk is the “pay for the lack of information”: 
in Table 8.4-4 r 21 = 1, r 24 = 6 (the payoffs a t j being 
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equal). To be sure, we would like to minimize the 
risk inherent in decision-making. 

We thus have two formulations of a decision-making 
problem: one seeks to maximize the payoff, the other 
to minimize the risk. 

We know that the simplest uncertainty is the sto¬ 
chastic uncertainty when the states of nature have some 
probabilities (? lt Q 2 , . . ., Q n and these probabili¬ 
ties are known. It is natural then (with all the perti¬ 
nent reservations in Section 2.2) to choose a strategy 
such that a corresponding payoff taken along the row 
is maximal 


n 

a i—^] Qj a u =*-max (8.4-2) 

Interestingly, the same strategy that maximizes the 
mean payoff also minimizes the mean risk premium 

r l = ^Q ) r lJ =^min (8.4-3) 

so that in the case of stochastic uncertainty both 
approaches (“from the payoff” and “from the risk”) 
give one and the same optimal decision. 

Let’s “spoil” our uncertainty a bit and suppose that 
the probabilities Q 2 , . . ., Q n exist in principle, 
but are unknown. Sometimes in this case all the states 
of nature are assumed to be equiprobable (the so-called 
Laplace decision criterion , or “principle of insufficient 
reason”), but generally this is not to be recommended. 
After all, it is normally more or less clear which states 
are more, and which less, probable. In order to find 
tentative values of probabilities Q t , Q 2 , ■ ■ ., Q n use 
can be made, for example, of the judgemental proba¬ 
bility assessment (see Section 2.2). Even these tenta- 
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tive values are still better than the complete uncertain¬ 
ty. Inaccurate values of probabilities of states of na¬ 
ture can later be corrected by means of a specially 
staged experiment. The experiment can be both “ideal”, 
i.e. completely clarifying the state of nature, and 
nonideal, i.e. relying on circumstantial evidence to 
clarify the state probabilities. Each experiment, of 
course, involves some expenditures, and the question 
thus arises: Are these expenditures compensated for 
by improved efficiency? It turns out that it only pays 
to perform the “ideal” experiment when its costs are 
lower than the minimal mean risk (see, e.g., [6]). 

We, however, will not consider the case of stochastic 
uncertainty, but will take the case of “unfavorable” 
uncertainty, where the probabilities of the states of 
nature are either nonexistent, or are not susceptible 
to an evaluation, however approximate. The situation 
is thus unfavorable for a “good” decision—so we will 
try to find at least not the worst one. 

Here everything depends on the approach to the 
situation, on the observer’s position, on the risks in¬ 
volved in an unhappy decision. We describe below 
some of the possible approaches, points of view (or 
decision criteria). 

1. Maximin (Wald) decision criterion. According 
to this criterion the game with nature is conducted 
as a game with an intelligent opponent (aggressive 
at that) seeking to hinder our way to success. A strat¬ 
egy is said to be optimal, if it guarantees a payoff 
in any case no less than the minorant value of the game 
with nature 

a = maxmina ii (8.4-4) 

i ? 

This criterion, being the “position of extreme pessi¬ 
mism”, dictates that one should always expect adverse 
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circumstances, knowing for sure that this is “rock 
bottom”. Clearly, this approach is “overcautious”, 
it is natural for those who are exceedingly afraid of 
losing. It is not the only one possible, but as an ex¬ 
treme case it is worth discussing. 

2. Minimax risk (Savage) criterion. This criterion 
is fairly pessimistic, too. It advices, in selecting an 
optimal strategy, to be guided not by the payoff, 
but the risk. As an optimum, the strategy is selected 
for which the risk under the most unfavorable condi¬ 
tions is minimal 


S — min max r tj 


(8.4-5) 


The approach dictates that one should by all means 
avoid large risks in decision-making. As far as “pes¬ 
simism” is concerned the Savage criterion is similar 
to that of Wald with the only distinction that the very 
pessimism is here treated otherwise. 

3. Pessimism-optimism (Hurwicz) criterion. This 
criterion recommends decision makers to rely neither 
on complete pessimism (“always count on the worst!”), 
nor complete, carefree optimism (“something may 
turn up”). According to this criterion the strategy is 
selected from the condition 

// = max [x mina i; - -f (1 — x) maxa w ] (8.4-6) 

i i i 


where x is the coefficient of pessimism selected between 
0 and 1. At x = 1 the Hurwicz criterion goes over into 
the Wald criterion; at x = 0, into the “complete op¬ 
timism” criterion that recommends one to select the 
strategy whose payoff in the row is maximal. At 
0 < x < 1 we have a happy mean. The coefficient 
X is selected from subjective considerations: the higher 
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the risk involved and the more security we are seek¬ 
ing for and the less we are inclined to take risk, the 
closer to unity x is chosen. 

If desired, a criterion may be developed that is 
similar to H and based not on the payoff, but on the 
risk, but we will not discuss it here. 

The reader may very well argue: if the selection of 
a criterion is subjective, the selection of the coeffi¬ 
cient x is subjective, too, hence the decision is taken 
subjectively, too. Where is science here? What has 
mathematics to do with all that? Would it not be 
better simply, without any mathematical tricks, to 
make a decision arbitrarily? 

In a sense the reader is right—decision making under 
uncertainty is always conventional and subjective. 
But still, to a certain (limited) degree mathematical 
techniques are of use here as well. Above all, they 
enable the game with nature to be reduced to matrix 
form, which is not always simple, especially when 
strategies are numerous (in our examples these were 
quite few). In addition, they allow one instead of just 
viewing a payoff matrix (or risk matrix), which, if 
large, may be simply a strain for the eyes, to introduce 
a consistent numerical analysis of the situation from 
different angles, to weigh recommendations coming 
from these angles and, finally, to decide on something. 
This is analogous to discussing a question from various 
stands, and a dispute, as we know, engenders a truth. 
Thus, do not await from decision theory some defini¬ 
tive, unquestionable recommendations. All it can help 
with is advice... ~ 

If the recommendations coming from various cri¬ 
teria coincide—all the better, hence you can confident¬ 
ly select the decision recommended—it is unlikely 
to fail. If then, as is often the case, recommendations 
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are at variance, use your brains. Reflect on these 
recommendations, and varied results to which they 
lead, finalize your point of view, and make the final 
selection. It is not to be forgotten that any decision 
problem involves a measure of arbitrariness, if only 
in building a mathematical model and selecting a 
performance measure. The mathematics used in op¬ 
erations research does not remove this arbitrariness, 
but only allows us to place it into a proper perspective. 

Consider an elementary example of a 4 X 3 game 
with nature whose payoff matrix [a^] is given in 
Table 8.4-5. We look closer at the matrix and try 

Table 8.4-5 



Ni 

n 2 

N a 

A 1 

20 

30 

15 

a 2 

75 

20 

35 

As 

25 

80 

25 

A x 

85 

5 

45 


offhand, without calculations, to decide on a strat¬ 
egy to be utilized. Even for the small matrix this is 
not so easy. 

We will now try to help ourselves using the criteria 
of Wald, Savage, and Hurwicz, and in the latter we 
will put x = 0.6 (a slightly pessimistic bent). What 
will they tell us? 

1. The floor is given to Wald. Compute minima in 
rows (see Table 8.4-6) and select the strategy for 
which the row minimum is maximal (25). This is 
strategy A a . 

2. Now, the floor is given to Savage. We will go 
over from the payoff matrix (Table 8.4-6) to the risk 
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Table 8.4-6 



N i 

Ns 

N 3 

(X; 

A x 

20 

30 

15 

15 

A, 

75 

20 

35 

20 

^3 

25 

80 

25 

(M! 

A, 

85 

5 

45 

5 

matrix (Table 8.4-7) and the 

right additional col- 

umn has the maximal value of the risk in a row, y t . 




Table 8.4-7 


Nx 

Ns 

N 3 

y i 

A . 

65 

50 

30 

65 

A-2 

10 

60 

10 

0 

As 

60 

0 

20 

0 

a 4 

0 

75 

0 

75 

Of the right 

column 

entries i 

the minimal one (60) 

corresponds to 

strategies A 2 and A 3 , hence both of 


them are optimal according to Savage. 

3. The floor is next given to Hurwicz (at x = 0.6). 
We again rewrite Table 8.4-5 but this time in such a 
way that in the right three additional columns we 
will have: the minimum of row a t , its maximum ojj 
and the quantity h t — xa ; + (1 — x) co* rounded 
off to the next integer (Table 8.4-8). 

The maximal value h t — 47 corresponds to strategy 
A 3 — a unanimous choice of all the three criteria. 
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Table 8.4-8 



Ni 

n 2 

n 3 

a i 


hi 

A\ 

20 

30 

15 

15 

30 

21 


75 

20 

35 

20 

75 

42 

^3 

25 

80 

25 

25 

80 

m 

a 4 

85 

5 

45 

5 

85 

37 


We will now take the case where there is a “conflict” 
between criteria. The payoff matrix t«£jl (with pre¬ 
liminarily written columns of row minima a;, row 
maxima to ; , and values of h t (at x = 0.6) are given in 
Table 8.4-9). 


Table 8.4-9 



iVi 


n 3 

N, 

“i 


hi 

Ai 

19 

30 

41 

49 

0 

49 

31 

a 2 

51 

38 

10 

20 

10 

51 

26 

^3 

73 

18 

81 

11 

11 

81 



By the Wald criterion the optimal strategy is A x , 
by the Hurwicz criterion (with x = 0.6) A 3 . We will 
now see what the Savage criterion will tell us. The 
risk matrix with an additional column containing 
row maxima y t is given in Table 8.4-10. 

The minimum in the last column is 38, so that the 
Savage criterion, just as the Hurwicz criterion, votes 
for it.,. 







252 


Ch. 8 Game Theory for Decision Making 


Table 8.4-10 



N, 

N , 

n 3 

N t 

Y; 

A , 

54 

8 

40 

0 

54 

A 2 

22 

0 

71 

29 

71 

^3 

0 

20 

0 

38 

m 


We should take a closer look at this. If we are very 
much afraid of the small payoff 11 associated with 
strategy A 3 , we will then select A x that is recommend¬ 
ed by the overcautious Wald criterion, whereby we can 
at least guarantee the payoff 19, and maybe even more. 
If our pessimism is not so black, we should, it seems, 
decide on A 3 recommended by two of the three criteria. 

The reader is sure to have noticed that here we are 
not speaking mathematical language, rather the lan¬ 
guage of consideration and common sense. That’s that— 
an uncertainty is no good and in the absence of ne¬ 
cessary information no mathematics will help you to 
decide adequately on an optimal decision. Such is 
our life, the future is full of uncertainty and not in¬ 
frequently we have to take not strictly optimal, but 
“acceptable” decisions, which are worked out using 
various approaches and criteria that appear as con¬ 
flicting sides. 

To sum up: the three criteria (Wald, Savage, and 
Hurwicz) were all formulated for pure strategies, but 
each of them can be extended to include mixed ones 
as well in much the same way as we did in game theory. 
However, in games with nature the mixed strategies 
are of limited (notably theoretical) use. Whereas 
against the reasonable opponent the mixed strategies 
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at times are of value as a “trick” to mislead the oppo¬ 
nent, in dealing with “indifferent nature” this argument 
doesn’t hold. Moreover, mixed strategies make sense 
solely for repeated games. But when we repeat it, then 
stochastic behavior of the situation is bound to stand 
out with the result that we can make use of it to handle 
the problem stochastically, which, as we know, yields 
no mixed strategies. Furthermore, in a situation with 
“unfavorable” uncertainty, where we need information 
so badly, the main task is to glean this information 
and not to engineer some tricks that would enable us 
to do without it. One of the major applications of 
decision theory is precisely to experimental design 
whose objective is to clarify and refine some data. 
We will not discuss the issues of experimental design 
here: it is a separate subject worthy of serious con¬ 
sideration. The interested reader is referred to special 
texts [29, 30], and also to a readable popular book [27]. 
The major principle of experimental design is that 
any prior decision must be revised with due account 
for fresh information obtained. 

* * * 


Thus, our short account of problems, principles and 
methodology of operations research came to an end. 
The author strove to introduce the reader not only to 
the potentialities, but also to constraints of mathe¬ 
matical techniques employed in decision making. 
But remember—neither of these techniques saves the 
worker the trouble of thinking. Not just speculate, 
but to invoke mathematical calculations, remembering 
that, as Hamming aptly remarked, “the major objec¬ 
tive of calculation is not figures, but understanding.” 
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OPERATIONS RESEARCH 

by E. S. Wentzel 


The aim of this book is 
to present in a widely 
appealing form the sub¬ 
ject matter and methods 
of operations research 
(OR), a managerial tool 
designed to increase the 
effectiveness of mana¬ 
gerial decisions as an ob¬ 
jective supplement to the 
subjective assessment. 
With no confinement to 
an exclusive area of 
practice, the discussion 
in the book is focused on 
the methodological as¬ 
pects which are common 
to all OR problems 
wherever they might ap¬ 
pear. Therefore the main 
emphasis is placed on 
such methodological 
treatises as problem for¬ 
mulation, model develop¬ 
ment, and assessment of 
computational results, 
rather than on mathe¬ 
matical rigor. 


In writing this text the 
author—a well known 
Soviet mathematician 
and popular novelist- 
employed her many-year 
experience in govern¬ 
mental OR projects and 
lectureship at the Mos¬ 
cow Institute of Trans¬ 
portation Engineering to 
give a vivid presentation 
of the relevant methods 
of mathematical pro¬ 
gramming, game theory, 
queueing models, and 
decision analysis. The 
mathematics of the book 
is simple and requires 
only expertise in prob¬ 
ability theory. The book 
is devoted primarily to 
the practitioners who are 
novices to the subject. 
It will be interesting 
to students, engineers, 
managerial staff and all 
those involved in deci¬ 
sion making. 


